TwitterFeed

[META] Activity Here

As you can see, I have started to update this blog more often than in the past… This is part of something I keep telling people who want to freelance/go indy, and I have had little time/motivation to carry on myself.

Being without boss means self-regulating every aspect of your life. When to work, when to relax and let inspiration come, when to immerse yourself in code despite all the distractions that are freely available to you.

Back when I started this blog, it was mostly a place to put some interesting tidbits about code I found during my projects that might benefit others, but over time, it became harder and harder to make snippets that would make sense outside any context. The various technologies I can credit myself for knowing intimately are not prone to simple 20 lines examples, and this led this place to have more and more empty time between posts.

Having gotten over the latest rush to have a product shipped, I decided to apply the discipline needed to work alone to this blog as well. I apologize in advance if some of the entries are bland – or completely uninteresting for that matter – but like with everything else, if you don’t practice something often and rigorously, you loose the aptitude and the momentum.

As a side note, getting motivated to do something is kind of a trick question in our line of work, as freelancers. It should be as simple as “I said I would do this” and do it. It might be hard sometimes, but put yourself a reminder that auto renews every so often days, and just do it. No boss means more freedom in terms of daily planning, but no one ever said it meant less work.

  

[HLT] Looking For Beta Testers

HLT2.0 is reaching RC stage. Please apply for beta testing!

If you haven’t bought HLT yet, I will give you a promo code, if you have (thank you) you will participate in the Circle of App Life : only requirement is to be able to submit bug reports in a clear way ;)

  

[MISC] Development Dance

Nothing new to developers but for some reason sometimes hard to grasp for the profane, the development dance is something that should be taught in schools.

In the mind of the non technical customer/boss, “development” is the process of going from nothing to a highlighted goal (specs, screens, that kind of things).

Most of the time, if you do a whole project with only developers attached to the process, for us code monkeys, it’s more like building something with Lego(s; bricks) : we build something that works, then iterate towards something like an ideal.

Cultural clash happens when the goal isn’t realistic from the point of view of the dev, but is non negotiable from the point of view of the non dev.

Which leads to the Dance : we take a few steps back on the goal, then a couple of steps sideways, then a few steps forward again.

Non technical people have to understand that’s the only way they can get close to what they want. Because the technical side is fraught with pitfalls, unwanted deadlines, miscomprehension, etc. Many a good feature started as a “what if?” coming from down below, only made possible because the foundations allowed them.

Going for a strict top-to-bottom approach gives you apps or websites riddled with bugs (because, as hard as you try, users will always use your work in a way it’s not supposed to support, or that you didn’t think of), and a strict bottom-to-top approach gives apps that are bland or ugly or without global vision.

Only a smart collaboration of the two can result in a decent product. Agree on key features, on a way the program should be handled in a way that’s considered normal, and fill the blanks as you go. That way, devs can geek out and build foundations that will handle anything and everything that they can think of, without having to fear you will change your mind once the hard work is done, and the non-techs can leverage the tech savvy to get something that will satisfy their itch of a beautiful, sensical product.

As with any relationship, concessions have to be made, and so far, the dominant culture has mostly been that developers don’t – and shouldn’t – have any input on the project, that developers are executants. What more companies get in return is a passive aggressive stance of “I’m going to do exactly what you asked, so that you can see it fail miserably”.

  

[WWDC14] Thoughts

I won’t go into details, the WWDC keynote has been covered far and wide.

  • New Look : √
  • New APIs : √
  • New ways to do old things : √
  • New Language : errrrr √

Response among the community was unanimous, this is xmas come early. And it’s true that for us developers, there a lot to be excited about. The new “official” way to communicate with other apps through the extensions mechanism is awesome, the integration of TestFlight will make a lot of things easier, especially for us small teams, and the new language will hopefully make us more productive (yay, less code to write).

There are some blurry or grey areas about these changes that will probably cause some problems, but hey, we’re Da Dream Team, right? We’ll manage.

The only thing that struck me as a slight cognitive dissonance is the fact that outwardly, Apple publicly recognizes our role in the success of the platform (huge), but kind of changes nothing in the way we are treated. I am definitely not asking for exclusive access to the thought process of Apple regarding what’s secretly being working on, I think opening up betas to pretty much everyone defuses the rumor mill, and might help get better .0 releases.

Since we are the people who make the “normals” want to get an iPhone/iPad, why is it so hard to have any handle on how we do it?

Xcode tends to get better, but there is still no way to expand its capabilities, or adapt it slightly to the way our brains handle code-writing. Third party IDEs (like AppCode for instance) that may not be perfect by any stretch of the imagination, yet still give us more flexibility, have a hard time adapting to the internals of the build process. We still have proprietary/opaque file formats for vital parts of the development (I’m looking at you XIBs and CoreData models). Cocoapods have become mainstream, but are still iffy to integrate (and might break).

For the social side of things, since WWDC is harder to get to than a Prince concert, same deal, it’s Apple’s campus, or community based (read no help from Apple whatsoever) things. Kitchens? Local dev events? Access to labs? If you’re not in California, tough luck.

So, yes. We are the main booster for the success of the platform, but we have absolutely no handle on things, in any way, shape, or form.

Am I excited that we get shiny new things to play with? Sure. Is my head buzzing with ideas? Yup.

But I am also a bit bitter that, sometimes, it feels like we’re not working together.

  

[HLT] Version 2.0 in the works

I have been so overwhelmed with work and system changes that it’s been hard to keep up with the Highlight maintenance.

But I’m pleased to announce this is officially over, I stopped slacking and a 2.0 is reaching the Release Candidate stage.

New features include on-the-fly screen changes detections, and text input (yes, you asked for it, you will have it)

Bug fixes include retina weirdness, and global shortcut fixes for Mavericks.

If you have some last minute requests or bugfixes to include, don’t hesitate to fill in a note in the bug/feature tracker

  

[Internet] The “Weird” Socio-dynamics Of Online Communities

I must admit, I’m a lazy dinosaur in this tech stuff. After one too many discussion about my relationships with people I don’t see very often (read less than once a year), I just feel like writing things down and pointing the next inquiry in that direction…

Dinosaur?

Yes, I used to hang out on IRC a lot. I still have friends I consider very close that I met online “back in the day”. I met people in my travel who I must say I am not keeping in touch with to the level I would like. But these are the most enduring relationships I have, to this day. Feeling closer to a random dude I’m having a conversation with on Twitter than to a random person I meet in a bar really is nothing new.

OK, so?

There are two main facets to the online relationship you might form with somebody. Most people who have little to no experience with that will focus on the negative side : it can be completely anonymous and the “person” you’re talking with may have little overlap with the human behind the keyboard. Extremes include complete impersonation (sex change, age change, skills change) for fun or profit. But the more positive thing is you can also often see the “real” side of someone, who’s hiding it most of the time because of his/her real life constraints. That includes the kid who’s so shy he wouldn’t even come to a bar, but can be Ragnarok-The-Viking-667 online and be really funny, knowledgeable and socially inclusive online. Depending on the kind of activity you are having, the drive to be a predator, either literally or not, can be made irrelevant. Some services and some games reward competition and jackass-ery online, giving an edge to the “evil” side of the coin over the benign one, but it’s far from being the norm everywhere.

Do you have data that supports this? numbers? because I had the totally opposite experience!

Not really. I guess I’m lucky or something. But I found that even in online games, within the confines of a group acting together, the tendency to appear bigger and stronger than everyone else fades after a while… Or maybe I avoid being in activities that encourage being solo, the top dog, or whatever. The thing is, in my experience, I tend to bond easily with people I’ve only ever seen type, or heard the voice of, some of which even became good friends and had or still have “real life” interactions with me.

When you think about it, “anonymity” or a partial one is key to a good therapy. You are not supposed to know your therapist beforehand so that you can talk freely to them. The kind of relationship that develops after that is up to the participants, but the initial point of entry is no assumption. And that can be very liberating to people who might feel left out for reasons of their own. Online, you can be whoever you want. Most people will settle down to be “themselves” after a while, which is not the social mask we have to wear in some real life circumstances. So why not give these relationships the shot they deserve? It’s not any weirder than forming a bond with someone you just met at work, in a bar, or a friend of a friend.

PS: that’s what “being lazy” means to me : you do it once, and then you don’t have to do it again ;)

  

[BTLE] Nike is probably going to kill the Fuel Band

During the weekend, I came across a CNET article reporting Nike will kill one of the major BTLE devices out there.

what does it say? (aka tl;dr)

Mostly that due to an opinion that the fitness sector is already kind of saturated, and some managerial issues, and the fact that Android Wear and related rumors on the Apple side will bear fruit soon-ish, Nike isn’t interested in hardware dev anymore.

what doesn’t it say?

It’s kind of hard to say what’s really going on. Nike isn’t some small startup who might fold overnight. The allegations of gross mismanagement are hard to verify. The tidbit about focusing on software and making a deal with Apple, since they will be making the phone/watch hardware anyway is interesting but misses a crucial fact: usability. Your phone does a heck of a lot already, and I have doubts it will do everything in the future. And fitness bands are unlikely to be replaced by a very costly and somewhat bulkier “iWatch”, if only because the physics and comfort involved will limit its capabilities in terms of tracking.

why do you say that?

Step counts? sure, a spike on the vertical axis of the accelerometer, provided you don’t move your arms around too sharply. Pulse, temperature, etc? Why not as long as it’s stuck to your skin.

But a lightweight device that needs to be recharged every week or so has a magical ability to become invisible. As such, it can track real habits. A thingie with a distracting screen? Less likely. If its weight or discomfort during the things it’s supposed to track is too much, the data will be even less reliable.

I’m a big proponent of specialized gizmos that do one or two things, but do them very well.

  

[Game] Kerbal Space Program

So true (applies to time I spent reading/playing with physics simulators, too):

XKCD -KSP

KSP is available here, but don’t click if you fear you might become addicted…

To the Mün and beyond!

  

Bluetooth, or the new old technology

Connected devices are the new thing. Thanks to the growing support of BTLE, we now have thingies that can perform a variety of tasks, and take commands or give out information to a smartphone.

When I started a project that relies on BTLE, my first instinct was to take a look at the documentation for the relevant frameworks and libraries, both on iOS and Android. According to these documentation files, everything is dandy… There’s a couple functions to handle scanning, a few functions/methods/callbacks that manage the (dis)connections, and some read/write functions. Simple and peachy!

Well… Yes and no. Because of its history, BTLE has a few structural quirks, and because of its radio components, a few pitfalls.

The very first thing you have to understand when doing hardware realted projects is that there a huge number of new (compared to pure software development) things that can go wrong and have to be taken into account: does it matter if the device disconnected in the middle of an operation? If so, what kind of power management strategy should you employ? The failsafe mechanisms in general become highly relevant, since we’re not in a tidy sandbox anymore. Also, hardware developers have a different culture and a different set of priorities, so an adjustment might be necessary.

Bluetooth is an old serial-over-radio thing

Yes. Not only is it radio, with all its transmission potential hiccups, it also is a 20 years old technology, made to have a wireless RS-232 like protocol. Its underpinnings are therefore way less network-ish than most remote-toting APIs. There’s very little in the way of control mechanisms, and no guarantee the packets will arrive in a complete and orderly fashion.

As far as I can understand it, it’s kind of like AT commands on a modem, and therefore prone to errors.

On top of it, BTLE adds a “smart” layer (smart being the name, not a personal opinion), which as a very singular purpose: syncing states.

Again, I see it from the vantage point of having a few successful RC projects under my belt, not an expert in the BTLE stack of the OS or the device. But as far as I can see, BTLE is a server over serial connection that exposes a hierarchy of storage units. These storage units have contents, and the structure of the storage, as well as the bytes it contains are periodically synced with the ghost copy in the other end of the link.

So, a device handles its Bluetooth connection as usual (pairing, bonding, etc), then exposes a list of services (top level storage containers of the tree, akin to groups), each of which containing characteristics (a storage space with a few properties, a size, and a content).

In theory, once you’re connected to the device, all you have to do is handle the connection, and read/write bytes (yep, bytes… No high level thing here) in characteristics, for Things to happen.

As you can see, having a good line of communication with the hardware developers is a necessity: everything being bytes means being sure about the format of the data, and knowing which characteristics require a protocol of their own, as well as which writes will trigger a reboot.

All in all, provided you are methodical and open minded enough, it can be fun to figure out a way to copy megabytes of data using two characteristics that can handle 16 bytes at the most. Welcome back to the dawn of serial protocols!

BTLE is all about data sync

Since the device is likely to be a state machine, most of the APIs mirror that: the overall connection has disconnected, connecting, connected, and disconnecting states, and synchronizing the copy in-system of the data with the copy in-device is highly unsynchronous. Not only that, but there’s no guarantee as to order in which they are transmitted, received, or timing thereof. You are a blind man playing with the echo, here.

If you want to transmit long structured data, you have to have a protocol of your own on top of BTLE. This includes, but is not restricted to, in-device offset management, self correcting algorithms, replayable modularity, etc etc.

Not to mention that more often than not the OS tells you you are disconnected after an inordinate amount of time, sometimes never. So the constant monitoring of the connection state is also paramount.

Last but not least, background execution of said communication is patchy at best. Don’t go into that kind of development expecting an easy slamdunk, because that is the very best way to set some customer’s house on fire, as the thingie you are managing was stuck on “very hot” in the middle of a string of commands and you never realized this was the case.

The software side, where it’s supposed to come together

Let’s imagine you mastered the communication completely, and know for sure the state and values of the device at any given time. First, let me congatulate you. Applaud non ironically, even.

Conveying the relative slowness of serial communications in the age of high speed communication in a manner the user will a/ accept and b/ understand is not a small feat whatsoever. It takes up to 4 minutes for us to push 300k of data in mega-slow-but-safe mode. Reading is usually 10 times faster, but we are still talking orders of magnitudes above what’s considered normal on these devices for these sizes.

One trick is to make the data look big: plenty of different ways to visualize the same 12 entries, another is to make it look like the app is waiting for a lot of small data points to compute a complex looking graph. All in all, it’s about making the user feel at home with the comparatively large “doing nothing visible” moments.

What I am hoping for the (very near) future

Bluetooth has always felt kind of wonky and temperamental. It actually has a very stable radio side, the problem largely lays in the total absence of the kind of control structures protocols like TCP uses to recover from errors, or slow and changing connections. At its core, the whole system seems to be built on the assumption that “once the connection is established, everything will be alright”. A lot of effort has therefore been put on discovery and power management issues, rather than any kind of self-correcting way to talk. It is a very complex system intended to establish a serial connection, then a level of abstraction on top of it in the form of a “server” that organizes the data in a somewhat discreete fashion. And that’s it.

If changing the protocol is too hard, and I’m totally ready to assume that it is, then the API providers need to figure out a way to manage the various states in a way that represents way more the reality of the transmission. Otherwise it’s a permanent struggle to circumvent the system or to coax it in doing what you know is what should happen.

  

[iOS7] Woes Unto Me, For I Am Undone

While I’m waiting for a full restore of my iPhone that will hopefully get me to a usable state again, there’s a couple of things you should know about iOS programming that has subtly changed.

First off: I like the new interface. I like the new OS. So this is not a mindless rant.

[UIKit]

UIKit has never been threadsafe. Ever. This is why we got used to performSelectorOnMainThread back in the pre-block days, then dispatch_async. The non-threadsafety of UIKit is a total mystery to me, seeing that I can change text in labels, images, and the like, mark the views as dirty, and then the main GUI should take care of the updates however long afterwards. That being said, Android does the same, so there must be a technical reason I’m not seeing. However, Android doesn’t even let me change anything. It’s an immediate exception at runtime. UIKit, on the other hand, lets me do pretty much anything I want, then crashes with a cryptic message, when there is one. After digging and step-by-step debugging, I usually find that what I assumed runs on the main thread, does not, in fact.

But that’s the kicker, right here. Up till iOS6, some conventions (or habits to be more accurate) led me to believe that this callback or notification would always be called on the main thread, thus not forcing me to use a very performance-expensive dispatch block. And then… it changes. The app starts behaving erratically, when it doesn’t outright crash. And debugging that erratic behavior is difficult, because it’s basically a race-condition. It could work just fine on an iPhone 4 but not on a 5, and vice-versa.

For a new codebase, I guess the problem doesn’t arise as much, but for us developers who work on older projects, sifting through thousands of lines of code to figure out which one causes that is time consuming, to say the least.

Case in point : navigation controllers. They used to be somewhat thread-safe, pushing and popping being stacked and executed one after the other. Of course, every now and again, you would have to tweak a little bit, make sure some essential data was loaded before pushing the next one, but nothing illogical.

In iOS 7, it’s totally asynchronous and unstacked. And unless you setup a delegate to make sure you detect the end of the animation, you can end up in very dark places. As soon as you have a more complex navigation than simple tree walking, something that worked reasonably well till now becomes very hard to maintain.

Let’s say I have a custom navigation bar that reflects some general data (ie not just a title with the name of the current screen you’re at). Now lets imagine some scenario where when you change something in one of the leaves of your navigation tree it requires a change in another leaf, where the user has to input something. You can say, for example, that you have a weather app, and in one leaf you have the general area of interesting data (wind, hygrometry, whatever), and in the other the units you want to use. Changing from one to another requires a change of units. So, naturally, you want to pop the view controller and push the units one. Except now, you can’t do it too fast. You have to wait for each animation to complete before taking the next action. And you really shouldn’t try to change the title while the animation is running either. (yes I know you could set the navigation hierarchy directly, but my rebuttal is the same as the one given afterwards)

And I hear someone say “well it’s easy, you have the two delegate methods for the navigation controller, so just use it”. Yeaaaaaaaaaah. Weeeeeeeell… The nav controller delegate has to be known at a higher level than the views you will push and pop, right? So basically, the app delegate would be the nav delegate. So the app delegate has to know about the underlying structure and the navigation paths of the whole application? Sorry, I do object oriented programming, I have no intention of having all my view controllers as instance variables of my delegate with a huge switch every time there is pop animation to determine whether I should wait for it to finish or not.

What would be a good alternative then?

  • If I had a notification I could subscribe to, that’d be swell. And that’s what I implemented at the higher level. But it’s a hack.
  • There could be a lock, too. [UINavigationController waitForAnimationToEnd] for instance.
  • Or a bool telling me whether the nav controller is in transition or not.
  • Hell, even an exception would be better than just putting the app in an uncertain state. At the very least I could break on it to find which one of my thousands of lines should be scrutinized.

Going the multi-app, multi-threaded, asynchronous, route, is fine by me. But we have to have the tools to do it properly. Even viewDidAppear is not a guarantee that the animation is done and changing the title or pushing a new view controller on the stack won’t give us a nice Finishing up a navigation transition in an unexpected state. Navigation Bar subview tree might get corrupted.. What did I do wrong that was perfectly fine up until 3 weeks ago? No idea. No exception. No information. Just “hey don’t do that”. Totally Kafkaian as far as I’m concerned, since I just hit on the back button. But now, after this point, any and all navigation related code will crash. And I’ll spend a few hours or even days to figure out why.

[Networking]

For the same reasons, networking (low level networking mostly, but with stuff creeping up sometimes higher) has changed. I can have network connectivity, tested, working fine in Safari, and a socket that connects to nothing while not timing out. Why? How?

Because that, too, has been moved to a new asynchronous mechanism that offers very little help. My NSURLConnection might appear to be doing nothing, but be active enough so that the system as a whole doesn’t deem it a timeout. And since I have no way of peering into its current state, apart from waiting patiently at the delegate points, the net result is that the app looks as if it’s stuck. Which it is, but not UI stuck. Can’t get out of that mode any other way than kill/restart for now.

I wish I had dug enough to give you more information about it, on par with the above UIKit problem, but the truth is it’s been 2 weeks and I still haven’t had the time to figure it out properly. And I have very little hair left. And it’s all grey.

[Conclusion]

I am for all the new changes. I like them, think they make sense, and overall improve and expand the possibilities for us developers. But without insider information as to how the thing works under the hood, we have very little ways of tackling these kinds of bugs. And all of them feel… hacky.

By all means, change the APIs, improve them, etc etc, but also give us the tools to do our jobs properly. Everybody wins if the users are happy.

And I wish I didn’t have to kill Mail so often, which seems to be stuck on the same problems I have. I guess that’s a sign that I’m not alone in my struggle.