Ongoing mistakes by WP7 Marketplace Certification with Bing Maps

The process of WP7 app certification can range anywhere from being simply and quick – through to requiring mountains of effort, frustration and a really long waiting times in between.

The problem is that not only the testing is just completely random/inconsistent (you’ll see tons of apps being certified which clearly fail to meet requirements) – it appears a large amount of the testers are really struggling to understand the certification rules themselves – and regularly and invalidly fail apps which are perfectly OK.

To give you an idea of my experience with this – over the past 12 months – I’ve published ~25 apps in marketplace (each having several updates/versions) – so that’s upwards of 75+ Marketplace certifications.  

Out of these – a large number have come back as failed  (perhaps  ~30%) – occasionally some for perfectly good/valid reasons (which I appreciate) – but vast majority has been due to a vague/ambiguously worded certification requirement being misinterpreted by testers.

This is clearly not good enough – and makes the process of releasing and updating apps hugely problematic (and quite often renders me helpless to quickly address other bugs/enhancements to apps which users are asking for).

 

The ‘Location Services’ related Certification Rules

One certification requirement however has without doubt caused me the most grief – the ‘location services’ privacy policy (section 2.7) – resulting in more ‘failed’ certifications for me than anything else (I’ve lost count of exactly how many – but perhaps has resulted in 10-15 failures for me).

Taken from the Windows Phone 7 Certification documents :
 

2.7.2 : The privacy policy of your application must inform users about how location data from the Location Service API is used and disclosed and the controls that users have over the use and sharing of location data. This can be hosted within or directly linked from the application.

and

2.7.3 Your application must provide in-application settings that allow the user to enable and disable your application’s access to and use of location from the Location Service API.

 

While you’d think for this I could simply implement this right once – and then ensure same functionality is applied to all my future apps/submissions (thereby guaranteeing I never get failed for that reason) – this is clearly not the case.

The problem is that it testers simply don’t understand this rule – and hence each one applies it differently – ensuring that no matter what has been done to correct a previous failed certification – a conflicting interpretation from the subsequent tester will result in yet another fail.

 

Bing Maps vs Location Services

The root cause of the above certification rule being necessary (with my apps) – is due to using the Bing Maps Control (as below). 

Whilst using Bing Maps by itself has no bearing on the above rules (as you are not providing or querying the user’s location simply by showing the map) – the problem is that in order to position a map or place a drop pin – you need to pass in instance(s) of the  GeoCoordinate class (which is a basic struct containing longitude, lattitude and height).

As per all my apps – they already has the longitude/lattitude of these pins hardcoded in the app metadata – so at no point do I request nor need the users actual location – and each time GeoCoordinate is used by my app – I initialize it manually with static coordinates.

The gotcha is this GeoCoordinate class is located in the System.Device.Location namespace (in System.Device dll) – so you have to reference this System.Device in order to use Bing Maps with drop pins. 

This then quickly becomes a problem – as when you submit your app to the WP7 AppHub for certification – the automated ‘capability’ detection tool invalidly decides that your app is actually using ‘location services’ (when in fact it’s just using a basic class to pass in static coordinates).

When testers then go to certify your app – they then jump to the invalid conclusion that all the location services rules and additional requirements need to be followed (as per couple mentioned above). This is despite the fact that at no point is the app ever asking for nor utilizing the location services to obtain a user’s current location.

NB: While it’s possible that ‘internally’ the Bing Maps may access your current location (this is not something that is discussed or revealed in any documentation) – what’s 100% certain is that there is no ability for developers to control whether a user’s location is accessed by Bing Maps or not (as no such properties are provided). 

 

A recent example of how one of my apps was failed incorrectly.

As mentioned above – at no point do these apps in question (which use Bing Maps) – ever obtain or use the user’s location (at least not to my own code). Nevertheless – repeated certification failures on this point have led me to go completely overboard in ensuring that these requirements are met.

The most recent example was my app ‘NHL Fixtures 2011/2012’ (which like my other sports apps) – use hardcoded/static geo coordinates to plot the location of a sports venue on a Bing Map (and never get the actual user’s location).  

Last week – I submitted the first version of this app.  After 4 days it was certified and published to marketplace.  Then, a few hours later – an end user provided feedback that a couple of the internal hardcoded URL’s were pointing to incorrect pages (which were displayed in the hosted Web browser – several pages into the app). It’s important to note that while no actual ‘error’ was thrown – nor did the app ‘crash’ as a result – the information displayed was a different web page. 

As a result I quickly fixed these hardcoded URL’s and submitted an app update (ie. no code or functionality changes at all other than to show a different web page). In the tester notes (as I always do) – I explained the nature of the update and as a precaution – again pointed out all the ways in which I’d satisfied these above to location services related rules.

Today (4 days later again) – I received a ‘fail’ from marketplace (on above rules) – invalidly claiming that I’d ‘not provided a location services privacy policy’ and that location services ‘couldn’t be disabled from the app settings page’.

This could not be further from the truth – as these ‘in app’ screenshots below clearly show (from this failed submission).

 

Proof + Screenshots of the Marketplace Certification #fail.

Please see red circles for relevant functions.

Firstly an example of where Bing Maps is used in this app (NHL Season 2011/2012). As per comments above – the drop pin and map center locations are hardcoded into the app (and not derived from a user’s location) in order to show the following Bing Map :

 

To satisfy requirement 2.7.2 (requiring that a location services privacy policy is shown) – here is 3 x places links to the Privacy Policy Page IS SHOWN :

   

and here is the location services privacy policy page that Marketplace testers claimed ‘didnt exist at all’ (ie. the page all these links above go to when clicked) :

The link on this page ADDITIONALLY allows user’s to view Microsoft’s Privacy Policy (in a web browser) – as it’s their Bing Maps control which may/may not use the user’s location.

Similarly – for requirement 2.7.3 – the ability to disable location services (the only logical way – which is to disable use of Bing Maps) – the following setting exists in the ‘app settings’ page :

When above settings is disabled – the following behaviour happens when you go to view a Map (it doesn’t load/use the Bing Maps until you re-enable this functionality in settings) :

Just in case everyone misses it though – the following text is also present in the App’s description on Marketplace (and shown to users) :

 

 

Given the overboard and comprehensive nature these requirements were met by me – you really have to wonder how this app was failed by Marketplace. Short of completely removing all Bing Maps functionality from my app – there’s nothing further I could have possibly done (every ‘i’ had been dotted and every ‘t’ crossed).

What’s more gobsmacking is how a tester could be under the impression that no privacy policy was shown nor were settings provided to disable location services.

Seriously #wtf?? marketplace – I provided no less than 5 mentions/links to the privacy policy throughout the app and description – how could they possibly have missed this or the app settings page.

 

Actions Microsoft need to take to remedy this ongoing issue.

 

As mentioned this is an ongoing and persistent problem for me (and no doubt many other developers using Bing Maps or location services) – and after nearly 12 months of WP7 Marketplace being around – it’s painful and frustrating that it’s still not be sorted out.  

The ability to quickly push out app updates as well as new apps is severely restricted – due to the long turnaround times by Marketplace (with the weekend – it can take up to 1.5-2 weeks to get a release out if it gets failed).  This poses a real customer service problem for me – as I’m unable to quickly respond to actual bugs/problems in apps by deploying updates – and of course limits ability to correctly time app releases.

It’s clearly not a case of ‘occasionally getting a bad tester not doing their job’ – it’s an ongoing mistake (likely due to internal policy/training of testers) which will keep happening on regular basis.

As such – I’d really like to see Microsoft take the following actions to try overcome it :

1.  Explain to testers that apps using Bing Maps need the System.Device DLL in order to place drop pins. (and doesn’t mean an app is asking for user’s location). Also explain this results in the invalid ‘location services’ capability detection on XAPs.

2.  Fix the App Hub submission process so that it correctly detects location services capability. That means (in more technical terms) – it needs to look for calls to the ‘actual API methods’ requesting this  – rather than just seeing if app references System.Device (as it does now).

3. In future – fix the API so that either the GeoCoordinate class is not inside System.Device – or simply fix the Bing Maps control so their are native methods to set locations without this class. ie.    .setLocation(float longitude, float lattitude) – not .setLocation(Geoocordinate l)

4. Finally – please clean up the sloppy and inconsistent work being done by testers. The screenshots above show just how badly the testers got this wrong.  I can’t begin to imagine what sort of shoddy work was performed by this particular tester – but if some form of audit process was applied to failed apps (requiring someone with more experience to confirm the fail) – then mistakes would be fixed before they impact the developer and delay their releases/updates.

 

 

Anyhow – that’s my 2 cents on this topic. I hope Microsoft really does address these issues soon.