• Share to Speech for WP – Creating universal app experience

    Posted on July 10, 2014 by ivanadmin in General, Share to Speech.

    Overall

    The story begins with Share to Speech for Windows, a text to speech and listen later app on the Windows Store. I was thinking about a Windows Phone version from the start as it was surely the platform that needed least work to port to. But with arrival of Windows Phone 8.1 it seemed to become almost trivial according to some sources. And as app got some pretty good reviews (like the last one on WinBeta), and even award on contest organized by Microsoft, it was reasonable to put some more efforts to spread it on more platforms. Not to forget that it could significantly improve experience of using the Windows Store app.

    Read further for more details, but here is the spoiler: it is reasonable easy, however it is far from trivial.

    Backend

    Well this wasn’t really hard. I could really move most of the files to Shared project, and everything worked. However, not that there were no problems.

    HTML Agility Pack library doesn’t exist for WP. I have referenced its file directly and it worked… almost. It cannot load the document from URI. Luckily, you can get a string from URI with WindowsRT API, and then supply it to HTML Agility Pack. This took some searching over the internet and some thinking and testing, but overall – not too complicated.

    Also, as Share interface works somewhat different on Windows Phone and possibly also due to low processing power, some small things like when to close share page needed to be changed, and I guess this is a better practice anyway.

    Speech engine

    This is where trouble starts. In theory I could port speech engine to shared project, meaning that app could compile, but it was useless.

    OK, first one very stupid problem. On the Windows Store, you can use text to speech without setting anything in the manifest. On Windows Phone, you need to ask for using of the microphone (!). What is the worst, the microphone also exist as the capability in the Windows Store app, and you don’t check it. As you guess that this is the last place where you would be searching for a bug, you don’t fix such a problem in a minute unfortunately. So this is the first minus to Microsoft till now.

    Then the real problem. Windows Phone can play background audio only by using Background audio task. I must say that it is reasonable, as probably there is no other way to do it on 512 MB devices. But it leads to the trouble – what is the best way to make WP speech engine? I decided to start with copy and paste of code, as refactoring something wasn’t really too easy in this case (it would need to be a new RT library). And it turned out this was a reasonable decision as the code deviated pretty much – if it wasn’t I would find common parts and refactor them later.

    First let me say something about Background tasks on Windows RT: they suck. It doubles when it is an audio background task. IDE is just not aware of it in any way, and you have to connect everything manually in a manner that is really not too intuitive. And if you make some mistakes app will compile and IDE won’t complain, just your background process won’t work. So without any clue you have to try to find why it is (my problem was that I needed to reference that audio task in main project, though technically I haven’t used it’s code anywhere). Developers always point how more productive they are when they use Microsoft tools. But in this particular case, it is comparable or below the worse standards. Another minus for MS.

    Ok, now objective trouble again. As you have device with just 512 MB and you try to create speech from its background task, you have very little memory for it. You must go after very fine optimizations to ensure that it works. I ported back some of those optimizations to Windows Store app, but not all, some of them are just a necessary evil.

    Finally, you don’t have access to your data model (and of course view model) in the background task. And you need it. So you must find some solutions – or to port it to common RT library (which might lead to a lot of work depending on what you port, as some syntax needs to be changed), or to make an additional data model for the background task (which might be optimized for it).

    To say that this all was easy, no it wasn’t. It wasn’t quick either. But still many apps won’t have to port this, or won’t have as much problems at least, so this part is somewhat app specific.

    Main Page

    Somewhat good news. I had a snapped view of Share to Speech that was very similar to what I wanted to be the main page of the app, so I could just copy that part of XAML and related background methods, and though it required some adjustments it was a very good start.

    Now bad news. You don’t have Clipboard on Windows Phone 8.1 RT apps. It is really an incredible omission. While it is truth that most people will share what they want, some of them would surely copy and paste texts sometimes (especially as IE on WP doesn’t seem to want to share the selected text on the page). Not much to do except to say one more minus.

    More bad news, you can’t simply copy code for File Open Picker. This is due to low memory, as the app actually closes after opening the picker. As I have already omitted paste I decided to mark this “maybe later”. It is important to note that WP Office apps support Share, so this is really not necessary.

    One oddity – as I have decided that the AppBar shouldn’t contain any primary icons, I expected it to collapse to its minimum size. But it kept to appear in normal size whatever I did (although it was obviously both useless and ugly) until I set ClosedDisplayMode="Minimal".

    One more oddity – buttons in WP have some MaxHeight set by default. Again the last thing you would think of, it would be more normal for larger screen, but if you apply your old style you might find that something doesn’t work as expected.

    Acutally there’s even one more oddity – On Windows Store if element of ObservableCollection implements INotifyProperty and PropertyChanged is raised, it raises PropertyChanged of ObservableCollection. On Windows Phone it doesn’t happen, though it is extremely useful specially with some limitations in binding in Windows Store XAML. Well, you have to find work around, but it is sad that this is omitted.

    Rethinking the app

    The fact that you have less screen estate, less processing power and memory requires not to just port the app, but to rethink it for a mobile phone.

    So instead of the Now Speaking, To Listen and Library, there is just the Now Speaking list. It is to be auto managed, so that items that you have listened to disappear at some point (this is to be implemented soon).

    Also, Now Speaking page is added – as you don’t have as much info about speech on small screen as on the big one.

    Share page has three options: Speak now in the background, Add to Now Speaking list and Share to PC.

    On the other hand, PC app will have Share to Windows Phone command. When the Windows Phone app finds such a link, it downloads it in the background (so that you can use it later even without a data connection) and automatically adds it to the Now Speaking list. A true companion app.

    Obviously, those people that obtain Share to Speech will be able to get the WP version for free. However, as 3.99$ is not reasonable price tag for mobile app, there will be also a special phone edition that will be free with some limitations and in-app purchases.

    Conclusion

    As you can see, it is a great progress and even for this app that was probably one of more hard to port, it wasn’t too hard. However, some Microsoft’s omissions are really unexpected.

    Beta testers needed

    So, beta testers of Share to Speech app are welcome. The process will start in a couple of days, and anyone who wants to join can write me on ivan dot icin at labsii dot com.

One Responseso far.

  1. […] Ho scritto un post dettagliato sul mio blog su questo argomento (http://labsii.com/share-to-speech-for-wp-creating-universal-app-experience/). In poche parole posso dire che: in molti casi probabilmente lo è, ma non lo è nel mio. […]