HeMoSiBi™ - Her Most Significant Bit

New York commuting assistance software with speech-enhanced UI for Tablet/Desktop PC

Comment

Overview

Current project HeMoSiBi™ - Her Most Significant Bit (NY-2015) extends the capabilities of the original solution HeMoSiBi™ -2013, a finalist of the International Application Innovation Contest AIC-2013 by Intel Corp (link to the original submission article on Codeproject.com: HeMoSiBi™ ). Both applications, primarily intended for the Ladies in New York City address various aspects of rather eclectic Big Apple life style, in particular, shopping and visiting experience, and quintessentially important commuting issues.

MyImg1

Fig.1 Original application HeMoSiBi™ -2013, sample screenshot

MyImg2

Fig.2 Current release HeMoSiBi™ -2015, sample screenshot

Application has modular structure and implements rather intuitive speech-enhanced multi-modal User Interface (MMUI) providing ease of navigation and usage. It is comprised of several core functional modules:

  • Interactive NYC 'virtual' Map linked to NY Subway, landmarks and shopping centers data
  • NY City Trivia-quiz covering variety of topics, including City history, landmarks, cultural life, etc
  • Slide-show dedicated to Big Apple landmarks and major points of attraction

Instructional video on YouTube™ (see the image link) created for the original solution provides detailed descriptive material and operational guidelines. It demonstrates the application running on sophisticated All-in-One big screen PC equipped with Lenovo Aura™ UI (this form-factor is well suited for the installation in various public areas, including Subway stations). The application can run on any PC platforms (Desktop/Notebook/Tablet form-factors) in traditional mouse/keyboard and touch-screen operational modes. In addition to aforementioned online video, which requires Internet connection, the application implements concise Voice Instructions feature capable of running autonomously (i.e. in local operation mode without Internet connectivity).

Potential Audiences

Even though, as reflected in the app title, the initial development was focused on certain women-specific point of interests (in particular, popular shopping centers in NYC), the entire application is well-adapted for much wider audiences (in particular, novice transit riders, business commuters and tourists, etc.) and serves both practical and didactic purposes.

How it works

As mentioned above, this integrated application is comprised of three essential parts: NYC Subway data visualization module linked to the Interactive NYC Map, NY trivia-quiz and NY Landmarks slide-show.

NY Subway data visualization module

NYC Interactive geo-map with corresponding expandable Subway station list User control are linked to the application database, containing data tables populated from MTA Subway static data feeds. This mapping solution implements multiple customization options, like zooming, panoraming and view mode selection (refer to the check box labeled 'show details' in the upper-right corner of the map area). Its layered structure allows various layers to be added or removed dynamically. For example, User can select and add multiple NY Subway services (trips) and points of interests, like 'what to see' and 'where to buy' in the Big Apple.

Expandable Subway stations list contains additional important information. Station's operation mode is coded with icons: dark disk and circle icons indicate Full-Time and Part-Time stops; moon-eclipse icon corresponds to late-night operations; 'dark-light' half-circle indicates rush-hours stops in peak direction; fractured rhombus indicates rush-hours and weekdays stops in peak direction only. The second line starts with wheelchair accessibility information encoded with two check-mark icons corresponding to the North/South bound stops. Plane icon indicates Bus or AIRTRAIN to local Airport. Available transfers list is the last data item on the second line. When User click on any Subway Service icon, it expands its Stations list and the entire trip will be dynamically drawn on the map. Second click will close the station list and clear the corresponding line from the map.

NY trivia-quiz and slide show

New York trivia quiz is located in the left-upper corner of application window. Slide show module located below includes sample images of New York City landmarks and points of interests. Slides auto-progression can be turned on or off - notice the check box labeled with letter 'A'. The line at the bottom of the screen provides a bit of wisdom and fun.

Multi-modal User Interface

As mentioned above, the application implements multi-modal User Interface (MMUI) adapted for both PC keyboard/mouse and touch-screen, in particular multi-touch operations. Aesthetic border-less window with upsized control buttons in conjunction with innovative single-click (or single-tap) discrete resizing makes it well suited for use with any pointing devices less precise than traditional PC mouse. For example, zooming of the Interactive map can be performed using PC mouse wheel, or, alternatively, by multi-touch gesture in touch-screen equipped device.

Development approach

Application development adheres to the cornerstone principle of separation of programmatic concerns, namely: all layout and aesthetic enhancement achieved via WPF/XAML; data content is stored in SQL Database (SQL CE); the algorithmic middle-tier developed in C# and compiled into .NET assembly.

Coding Languages, Frameworks, Tools

.NET managed code base developed in C#; SQL/T-SQL used for database operations on Microsoft SQL Server Compact edition (SQL CE). Speech enhancement has been achieved via .NET 4.0 TTS object library/api. Interactive Map is built on Microsoft Bing mapping technology. Data content was obtained from public/open sources, in particular, MTA static data feeds (.csv data files in compliance to GTFS specs). Microsoft Access Database has been used at the development stage as a convenient data analysis/modeling tool.

Data Update

Static Data

At the development stage of this first production pilot, static data was imported into application Database in two-steps semi-automatic procedure. First, MTA static data for NY Subway, Buses, LIRR and MNR was imported (w/single mouse click) into Microsoft Access database file using csv-to-db data import macros written in VBA. For better performance, original data was inserted into normalized and properly indexed data Tables structure with multiple Table relationships and constraints added to ensure the overall data integrity. Secondly, the MS Access Database was converted into Microsoft file-based SQL Server (CE) integrated into the application. In a full-production release, the data update in application SQL server will be implemented with two options: as a fully-automatic schedule-based/event-driven procedure and as a manual single-click procedure with customizable data-import specs (note: at that production stage MS Access Database will NOT be used/required).

Dynamic Data

Real-time Bus tracking module (similar to MTA Bus-time app) was partially prototyped and tested, though certain issues prevented this part to be completed at this time (i.e. missing static stops data). In a full-production release this module will be extended with additional features addressing travel dynamic metrics such as fuel economy and commuting safety/comfort (in particular, by adding the innovative proprietary “vehicle driving profiler” solution - currently under development; proposal has been submitted to the NSF). This module could be also extended with crowdsourcing data aggregator, corresponding to the commuting metrics generated by aforementioned vehicle driving profiler.

Development challenges

  1. Original NY Subway data visualization solution in existing demo app (HeMoSiBi™ -2013) utilized very limited sample set of geographic information (pertinent just to "1" and "R" services), obtained from publicly-available online resources (wiki) and stored in XML data structure. Current release implements sophisticated relational Microsoft SQL Server Database with significantly extended data set imported from NY MTA static data feeds. As a part of development efforts, a database model and a set of semi-automatic csv-to-db data import procedures has been developed.
  2. MTA NY Bus data visualization module was planned for this app, but not completed: certain bus data is seemingly missing from MTA NY Bus static data feed - per the preliminary analysis 81 bus routes (out of 316) are missing some stops data. Development of this module is postponed until the issue was resolved
  3. The size of MTA Bus stop-times Table: corresponding to 8,855,843 records found for all NY Boroughs combined, it resulted in Access Database file size close to (and with added indexing - even exceeding) its limit of 2 GB.
  4. Data types compatibility issue: arrival_time/departure_time data items presented in MTA static .csv files using a string format like: 24:30:00, 25:xx:00, 26:xx:00, etc., even though in compliance with GTFS specs, are not compatible with DateTime data types pertinent to many popular strongly-typed relational Databases (in particular, Microsoft Access and SQL Server). If stored as a text (seemingly the only available option), it will limit the ability/performance of date/time calculations (suggestion: to modify data set to show just arrival time at first stops and use relative intervals for all subsequent stops, thus all data items will be type-compatible with DB fields)

Key Accomplishments

  1. Sophisticated speech-enhanced Multi-Modal User Interface MMUI) universally adapted for PC mouse/keyboard and touch-screen operations.
  2. Aesthetic border-less app window with unique one-click (or one-touch) resizing solution, providing additional operational convenience, particular important for User's with certain disabilities.
  3. Voice feedback (instructional, and notification messaging) providing additional operational convenience
  4. Powerful local application Database (built on Microsoft SQL CE technology) capable of MTA static data storage in a compact/normalized form
  5. Efficient set of SQL operations and algorithms pertinent to MTA static data sets
  6. Highly-interactive MTA Subway data visualization solution, based on data list linked to embedded geo-mapping Microsoft Bing™control
  7. Knowledge base and image gallery reflecting various aspects of NY lifestyle, presented in inspirational and aesthetic manner
  8. Speech synthesizer application software Pericles™ TTS-14 for Win developed by the Author and used to produce the voice-over narration in instructional video
  9. Real-time NY MTA Bus Tracking demo web app was prototyped and deployed to Azure cloud: REAL-TIME MTA BUS TRACKING ONLINE

Demo screenshot: Real-time MTA Bus Tracking web app (Route M1)

Learning points

  1. GTFS specs in general and practical working knowledge of MTA static data feeds in particular (Subway, buses, LIRR, MNR, SIR)
  2. MTA Bustime real-time application

Known issues and bug fixes

  1. Subway station data accuracy check: it seems not all transfers are included
  2. NY MTA Bus data: need to obtain complete bus stop list with geographic location info

What's next for this application

Current solution could be extended with several additional features/data sets:

  1. NY MTA Bus data visualization module
  2. NY MTA LIRR data visualization module
  3. NY MTA MNR data visualization module
  4. Traveled distance info
  5. Estimated trip time (potentially utilizing crowdsourced real-time/dynamic data feed TBD)
  6. Aesthetic enhancements, which should incorporate MTA IP (logos, symbols) - upon obtaining the license
  7. Additional points of interest data set linked to MTA Subway/Bus data and corresponding geo-location based search queries, like web apps developed by the Author: 'Find Tutor' or 'Find Job Agency'. Correspondingly, the "reverse" query would locate MTA Bus/Subway stops nearby the selected point of interest.
    8. .
  8. GPS-based real-time nearby Bus/Subway station and points of interests locator
  9. Commuting fuel economy and comfort/safety metrics added, depends on the anticipated outcome from proprietary "Ergometric Driving Efficiency Profiler" project tentatively code-named EnRoute-NY2015 (proposal has been submitted to NSF in the Y2014, currently under review) - TBD

The Author

This entire project app has been developed by Dr. Alexander Bell, president of the Infosoft International Inc (hi-tech consulting Company in the NY City). Alex is a Microsoft technology veteran with IT experience span from DOS to Win 8. He is known for his contribution to online development communities and participation in multiple software development contests, including aforementioned Application Innovation Contest (AIC-2012/13 by Intel Corp). Dr. Bell is a pioneer/enthusiast of green technology, AI/speech-enhanced Windows applications development and novel MMUI concepts. He is credited for 37 inventions and has received 9 "Best Design Idea" awards from EDN and Electronic Design Magazines. Beyond his professional duties, Alexander is devoted to digital photography, classical music and art. He holds a 'Legend' status on popular development site Codeproject.com.

Share this project:

Updates