[Xcode] IPA Generation Broken, Again

It happened before and shall happen again: xcrun PackageApplication doesn’t work anymore.

The whole process isn’t meant for contractors

Let’s face it, Xcode is getting better at the coding part, but the signature / IPA generation is “streamlined” for the App Store, which in turn is under a lot of assumptions that I’ve mentioned here in the past. Since the last time, the heavy push for storyboards (oh, the tasty git conflicts), and the weird quirks of swift haven’t really improved things. Dev tools should be as flexible as possible, because they are meant for devs.

Anyways, 8.3 broke the “easy” IPA generation that everyone was using. It was especially important for Hudson/Jenkins continuous integration and beta deliveries. No, relying on the Organizer’s UI doesn’t work, because more often than not, you want automation, and Xcode is less and less automatable.

PackageApplication is gone

It worked for a while, then you had to remove a part of the entitlement checking, but now it’s not there anymore. So… no more Hudson builds?

Fret not! The one-liner may be gone, but we have a heavily convoluted multi-step export process in place that does the same!
(Imaginary tech support person)

The idea is to replicate the UI process:

  • generate an archive
  • click export an IPA
  • select a bunch of signature and type options
  • rename all the things it exported (you usually want the build number/version right)

Believe it or not, some of these are actual xcodebuild commands. But the argument list is… errrrr, weird, if you’re used to Linux and server stuff. So, no, in the following, it’s not a mistake that there’s only one dash. You have been warned.


There are two pieces to the export thing : a plist containing what you would type/click in the UI options at the end of the process, and a script that chains all the necessary actions together. I tried to piece together various sources into a coherent whole, but it’s a dev tool and your mileage may vary. Also, because it’s intended as a Hudson build script, it takes the version number as argument and outputs what I would upload to the beta server in Outfiles. Again, adapt it to your workflows.

The plist:

< !DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">

Remember to use the correct TEAM_ID in there.

The script (takes the version as first argument, may break if none is provided):

APP_NAME="Insert app name here"
SCHEME_NAME="Insert scheme name here"
CONFIGURATION="Insert configuration name here"
#change as needed
echo "Cleaning previous build"
xcodebuild -sdk "$SDK" \
-scheme "$SCHEME_NAME" \
-configuration "$CONFIGURATION" clean
echo "Creating Archive"
xcodebuild -sdk "$SDK" \
-scheme "$SCHEME_NAME" \
-configuration "$CONFIGURATION" \
-archivePath "$ARCHIVE_DIRECTORY/$APP_NAME-$1.xcarchive" \
echo "Creating IPA"
#export ipa
xcodebuild -exportArchive \
-archivePath "$ARCHIVE_DIRECTORY/$APP_NAME-$1.xcarchive" \
-exportPath "$OUT_PATH/" \
-exportOptionsPlist exportIPA.plist
#name files and dSYMs
mv "$OUT_PATH/$APP_NAME.ipa" "$OUT_PATH/$APP_NAME-$1.ipa" 
cd "$OUT_PATH"
zip -r "$APP_NAME-$1.dSYM.zip" "$APP_NAME-$1.dSYM"
rm -rf "$APP_NAME-$1.dSYM"
cd ..
# rm -rf Archives

As usual, use with wisdom ;)

Final Thoughts

It’s not that the system is bad, per se. It actually makes a lot of sense to have commands that mirror the GUI. But the issue is, for a long long long long time, automation hasn’t been about simulating clicks. Since you don’t have the UI, it makes even more sense to have shortcuts in addition the GUI counterparts


Automagic Asset Generation For Apps

Between the iPhone (non retina, retina, 3x), iPad (non retina, retina, and pro), and Watch (2 sizes), generating an icon for all those things necessitates no less than 26 images. I have been using a custom Automator script to generate them all through a service that accepts any image as an input, which grew quite big.

As I was updating it, I thought it might be a nice thing to do to let my fellow sufferers have it. So, if that is something you think you might have a use for, feel free to grab it. As a matter of course, it works for me, but I make no promises for your workflow. Just drop me a line if you want to say thanks, make modifications you think might be useful for others, or throw curses at me for having released that just as you finished yours.

It uses no dependancies on libraries or custom applications, by the way.

Download XCAsset Automagic Generator right here!

Mood : working
Music : Griffin House

[Xcode] Broken IPA Generation

As of Xcode 6.0.1, you can only generate an IPA with a certificate/provisionning profile pair that matches a team you are part of (it offers only choices present in your Accounts preference pane).

Before ranting about why this this is stupid as hell, here’s a workaround:

xcrun -sdk iphoneos PackageApplication [path to the .app] -v -o [path to the ipa] --embed ~/Library/MobileDevice/Provisioning\ Profiles/[profile to use] --sign "[matching developer NAME*]"

NAME is the name that appears in the popup menu in your xcode build settings, not its ID

After a somewhat lively discussion on Twitter about that, two things:

  • I know Apple would prefer that developers are organized in teams managed on their servers. It’s just not practical for a lot of them, and there’s even a few good reasons not to go that way
  • It’s stupid to have that drastic a change, when 3 weeks ago, the official way of having a third-party developer generate IPAs for your company was to give him a p12 and a .mobileprovision and let him do his thing, and to not warn people about the change

For those of you who don’t know how development of a mobile application works, yet, here’s a quick run down.

A customer contacts me for a development. We agree on a timeframe and a price. I write the code, provide them from time to time with testable betas. When we agree it’s finished, I give them the final IPA for putting on the store and we call it a day.

Providing betas and giving an IPA for the App Store work exactly the same: a binary is produced, which is put in an IPA (kind of installer package for iOS), then that IPA is signed, and transmitted. On the other end of the wire (be it the customer or the App Store), the IPA is decompressed, the signature checked for validity (by app ID, device, and status of the apple account), and the app can be run or put on sale.

In that scenario, if I use my certificate, I have to enter the device IDs the customer will test the app on, of which my developer account allows for 100, in total. So if I have 10 customers with 10 devices a year, I can’t work anymore. So, most of the time, the customer has to provide the relevant information for me to give access to the betas, and of course, since they’re releasing it under their own name, the relevant information to produce the final version, which is a different pair of keys.

So far, so good, all they had to do up until now was give me a couple of p12 (key archives) and the corresponding profiles, and manage themselves the test devices, the release, etc.
It allows whoever’s in charge to retain access and knowledge about what the company is doing. Does that person want me to see they are also working on a concurrent product to something I’m doing for somebody else? Of course not. And there’s no reason to give me that kind of access. Oh and if the customer wants to prevent me from using that certificate again, all they have to do is revoke it.

The new way of doing things is to have the customer invite the developer in the team (in the Apple sense of the term), which gives the developer access to every piece of information under the sun (even when we can’t use it directly).

This is part of an ongoing cycle of making life difficult for contractors. We always have to find workarounds. The idea that almost every ios/mac developer out there is writing code for the structure they belong to, who will then release it in their own name for the general public is ludicrous. It hinges on something that has been gnawing at me for years: the idea that code and binary are the same thing, and is what I’m selling.

That idea is false. When you get Unity3D for your game development, you DO NOT GET THE CODE. For Pete’s sake, we don’t get the code of the OS we are developing on! The idea that when a developer is hired, the customer automatically owns the code is one of the many fallacies I have to deal with on a monthly basis. You hire a developer for his/her expertise first, and foremost. It might be then applied to internal code, which the dev won’t own in the end anyways, or to a newly minted piece of code which might or might not be given with the ability to use said code as part of something that has value. It is a separate item on the negotiation list.

I might delve into the pros and the cons of giving out your source code with the binary in a later post, but let’s get back on topic.

If, like me, you don’t always give the code with the binary to the customer, you’re screwed. Of course they won’t give you access to their company’s secrets by adding you on the team, if they don’t want to. And, obviously, you can’t release the binary under your own name for a customer who wants an app.

Please give me back a way to sign IPAs for customers, directly from my IDE.

Thank you, and sorry for ranting.


So… That’s How They Do Things Over There?

Every now and then, I have to fix up a project, generally huge and spaghettified. Don’t get me wrong, my own projects must look that way to other developers as well… The question is not about relative qualities of code styles and habits, it’s about getting in a boat steered by ghosts and guide it to calmer waters.

So how do you get to understand someone else’s code? We all have our own techniques, obviously, but I’ll list here a few that will help desperate people.


First off, run a Doxygen auto-documentation, with call and caller graphs. Doxygen is a freely available tool that you can download at this address, that can deal with many many languages. Make sure you have dot (available here) installed as well.

Go to the root of your project folder, type

doxygen -g

It will produce a Doxyfile text file in which there’s a bunch of options for generation. Edit it in your text editor of choice (I like BBEdit the most), and make sure the following options are set

HAVE_DOT               = YES
CALL_GRAPH             = YES
CALLER_GRAPH           = YES

Then run


Go make yourself a cup of coffee or something, it can be a long process. In the end, you have an html tree of files that gives you the class hierarchy, class documentations, and for each function, the caller and call graphs (what this function calls in the code, and what it’s called by). Ideal to identify the “major” functions called by a lot of others, and to evaluate the damage the changes you make will do to the rest of the code.

If for some reason, no code file is examined, you may have forgotten to tinker with the input section (folder in which the code resides and recursive search).

Alright, so, now you have a static and broad view of the code. It’ll help setting up the breakpoints in the bottlenecks and get a better idea of the overall architecture.


Now, the problem is dynamic. How does the program behave in time? To learn about that, depending on the language and the platform, there’s a variety of tools that can provide you with this information, but since I’m mainly a Mac/iOS coder, I’ll mention Instruments. Instruments is bundled with the developer tools, and you have nothing to set up to use them. Just open your project, and in the “Product” menu, run “Profile”.

Now, there’s a bunch of types of profiles you can get. What matters most to the project is dependent on what you’re supposed to do… If it’s “making it speedy”, then you want to go for the time profile, which will give you the percentage of time spent in various functions. If it’s a memory optimization, you have the object allocations and leaks tools. Etc etc… I won’t write a manual up here, you can find a relatively thorough documentation on Apple’s website.

The important thing is that for these tools, you will get the functions the program spends the most time in, that use the most memory, that leak the most, that handle the most complex CoreData queries, or whatever else strikes your fancy. In other word, you’ll get a better picture of how the program behaves in time.

It takes a little practice to identify what’s mostly relevant from what’s highly relevant, but it’s less of a daunting task than figuring it out through printf, NSLog, or NSLogger

Tying it up together

To take an overused metaphor, you now know where the engine(s) is (are), and how the beast handles on the road. Make sure you take time to study the bottlenecks, both in the structure and in the profiles you built. That way, tinkering with stuff will be comparatively easier.

If the structural and temporal (or memory-usage) bottlenecks overlap, you have your work cut out for you. They should be the focus of your attention. If not, then it’s a little trickier, and vastly depends on the perimeter of your mission.

Just try to remember something: most of the time, the developer who wrote the code had a different set of constraints than your own. While it feels good to rant at these dimwitted douchebags, it’s not fair. Most people who would have to take over your code would say the same.

Coding is a matter of education, style, and constraints/objectives. If you switch any or all of these parameters, you end up with very different code blocks. Understanding how it works and how to fix it should take precedence, and ultimately provides the best feeling ;)


[Me Too!] Xcode 3 vs Xcode 4

Amidst the flurry of trolls and meaningful comments about the rather drastic changes from Xcode 3 to 4, I’d like to sum up a few of the major discussions out there, and add a couple of my own remarks

First things first: a word of caution

This is not and should not be about simply changing habits. Yes it sucks to have to re-learn something you’ve used daily for nearly a decade, especially if it means wasting time you need to earn a living. But in the business of making software, we convince potential customers of much much worse. So if your main gripe about Xcode 4 is “dude, I used to do this like that and they are forcing me to do it differently” but the end-result is the same, please abstain from commenting.

Xcode is the only way to make iPhone/iPad/Mac applications. Yeah yeah yeah, you can have makefiles instead, in the same way I can run my car with whiskey: it works, but it borks pretty much everything and might end up blowing the whole thing apart. Unless you are Apple and know exactly the side effects of your build process, you have to use Xcode.

And of course that’s partly why it stings. As mentioned previously here as well as elsewhere, making an app is more akin to art than science. No one would tell a painter: “make a portrait of me, using only a 4-colors ball pen”. Some restrictions are funny, but they can also be ridiculous. By forcing us developers to adapt to Xcode instead of working out how to adapt to us, Apple is being kind of condescending. They know better than us. But that too is not the object of this post, so just leave it aside as there’s no alternative, and that can’t be a meaningful debate.

The Cardinal Rule: You Shan’t Leave

Apple has put a lot of effort into this: if you’re doing it right (for a given value of right, obviously), everything is done within Xcode: version control, file creation, code/xib/coredata editing, debugging, archiving, publishing to iTunes. There’s only one exception to this: Instruments, for profiling your app, finding leaks etc. I wouldn’t be surprised if there was a technical reason for this, probably permission-related, but I bet they are doing their best to integrate it rapidly. A lot of IDEs work the same way, and it’s not necessarily a bad thing.

I just have a thing against mono-window interfaces, I like all my important files to be open in different editors so that I can switch back and forth rapidly. Putting everything in the same window suggests every file has the same importance, when I just know it’s not the case… And I’ll be damned if I understand the new counterpart system: I used to have a small box in the corner; when I clicked on it, it switched from the .h to the .m and vice versa. Now, I have keyboard shortcuts instead, and most of the time, I get it wrong.

The mono-window allows us to see all the projects we are working on. That’s pretty cool. I’ve been using workspaces for this for a while now, and it’s nice to integrate it into Xcode: my 3 projects for this customer are together and the 2 personal ones are in another workspace, etc. As a result, the file filter at the bottom becomes really useful: I don’t have to spotlight all my disk to find where in hell I’ve done that gimmicky thing again: since my projects are grouped by customer and/or kind and/or whatever criteria I want, on a day-to-day routine, it’s a lot easier to find what I’m looking for. Swell!

The mono-window has one major drawback though: I’m not always at home on my 27″ screen to write code. And my 13″ laptop screen is way too small to use Xcode comfortably. Editing text can be done, editing a xib with its relevant .h open on the side requires a lot of juggling.

And that brings us to the major side effect of putting everything in the same window: UI editing. Everyone I know agrees it’s the best thing since sliced bread. Good bye Interface Builder! You have your interface in one editor, the relevant .h/.m files in (an)other(s) and the famous ctrl drag (aka the right click drag) works directly on the code. You’ve put a view in there and you forgot to add an outlet? Ctrl drag the view on the .h, and Xcode offers to insert the outlet in the .h (optionally as a property) and puts the standard code for unloading it properly in the .m. You can obviously connect it to an existing outlet, but many-a-time have I forgotten to put it in the .h and now I am saved! Getting used to the sidebar inspector might take a little while, but in the end it’s a success. Unless, of course, I am on my laptop. Never being able to see the whole view you’re designing just sucks.

So, apart from the fact that I don’t like mono-window for personal reasons, it’s pretty much a success on that part. However, since I’m not supposed to leave Xcode for any reason, it would make a lot of sense to have a plugin mechanism. Yes, I’ve been saying that since revision 1.0 of Project Builder. Back in the day, I wanted to use the Metrowerks compiler, which I found vastly superior to GCC (and apparently I was right, since Apple pretty much switched to LLVM, not wholly abandoning GCC, but clearly saying it’s not the best compiler out there). It took some time to figure out a way to hack together an “xml plugin” to add it to the list of available compilers. Or maybe I have a set of data I will embed in the application that I’ve got a different editor for that I’d like to integrate. Or I just want to do geeky things. Any and all professional application should have a plugin mechanism, bar none. Xcode should not be the exception. Because pros have sometimes very specific things they need to do, we need to be able to tweak Xcode to grant us that power. Otherwise, we have to get out of Xcode to get the work done.

The Way Of The Developer: It’s All About Flow

The typical work cycle of a developer usually includes the following steps:

  • Pulling the last changes from the revision server
  • Making sure everything still works after the pull (that means running and/or debugging)
  • Editing code to fix bugs, add new features, or because the customer/boss wants-it-done-dammit
  • Running/Debugging
  • Testing for regression
  • Committing your (working, hopefully) changes to the revision server
  • Publishing the result: that includes sending a beta to relevant people, putting your application out there, …

The first step is getting a lot better in Xcode 4. I still can’t figure out why Xcode doesn’t offer me automatically to pull my SVN changes from the server, sometimes, but it kind of works now. I still like to use the terminal for that step, but at least I know I don’t have to force the people I work with to do the same.

I’ll talk about running and debugging a little further down. I promise.

The edition part, provided you have a big screen and have learned to navigate around the iTunes-y interface, is pretty much covered. Of course, I’d like a plugin mechanism to download crash reports and integrate ticket-management right into xcode, but hey… Maybe someday.

For me, the first gripe starts with running/debugging. First of all, I have to select the right scheme AND destination (Remote Tickets in the Simulator, as opposed to the device). This forces me to do some heavy duty gymnastics, especially when the bug I’m working on doesn’t have the same results on the different destinations (I’m looking at YOU iPhone). How many times have my trusty co-workers heard me yell obscenities because for the tenth time today I ran the program in the wrong environment? And they still jump from their seats to this day.
The side-effect is that depending on this root choice, some actions will disappear. I’m running the simulator, everything’s a-ok and I want to push an IPA? Tough luck, I have to switch to the device environment just for that. Tell me something Xcode: why in hell would I want to archive a simulator version? Therefore if I want to archive a binary, it will automatically be for the device. Therefore the menu should always be available as there’s no alternative!

Testing for regression is also a lot harder than it should be. For Pete’s sake, I have different versions of the simulator (one for every iOS device, running every supported OS version), and I have archives of my builds. Why can’t I keep them somewhere and run them for comparison without having to use the SVN/git/whatever branches? That forces me to have at least 2 versions of the same project open at any given time during my tests.

Committing still has its quirks, but all in all it works fine in Xcode. Finally.

And the last part is so ridiculously convoluted that it deserves a chapter of its own.

Publishing An App (aka Russian Roulette)

Let’s be serious for a second. The objective and benefits of publishing an application isn’t the same for the developers and for Apple. Therefore, there can be some goodwill on both sides, but, as developers we don’t work for them and they don’t have that many incentives to work for us either. Therefore, I’ll just go over that whole certificate thing very quickly.

Apple’s business model is that if you want to distribute an application, you do it through them. Period. There can be some leniency here and there, but that’s the bottomline. To ensure that, we are issued temporary permission to run our work on development devices (ours or our beta testers’), through the use of certificates. A “provisioning profile”, or a certificate in more trivial terms, is a combination of 3 permissions:

  • You are a certified developer (aka you have paid), therefore you have a right to run your work on devices
  • The device is declared in a list of devices authorized to run stuff from you
  • The stuff you are running is (at least partly, for identification purposes) declared in a list of stuff you are doing

This mechanism protects Apple from a black market of applications that wouldn’t go through them.

There are two major types of provisioning profiles: testing/development ones and distribution ones. Development profiles can be used to test your software on devices. Therefore, you are only authorized a limited amount of test devices. Distribution ones are unlimited in devices, and can be issued for the App Store (distribution by Apple) or for Enterprise distribution (a more expensive and legally binding contract).

In my work, there are 3 typical scenarios:

  • I work on a personal project that will be released to the general public (via the AppStore)
  • I work on a project for a company that will publish the result on the AppStore
  • I work on a project for in-house deployment

In Xcode, in the project settings, you set the provisioning profile you are going to use, and at the end of a successful build, the application is signed accordingly. Then you’ve got to figure out a way to deploy it.

In the first case, it gets easier with Xcode 4 than Xcode 3: Publishing an application to the AppStore is ridiculously easy nowadays. Just build and archive, Xcode makes sure all the minimum requirements (icon size and so on and so forth) are met, and pushes the new version of your application straight to Apple for evaluation. Since my own test devices are included in pretty much every profile I have on my computer, testing for myself is dead easy, I don’t even have to change the settings: any certificate will do. At distribution time, a popup asks me to confirm the distribution profile I want to use anyway, so why would I do anything about it? Congratulations on a job well done.

Where it gets hairier, is when you have beta testers. To test your software, they need to have given you the ID of their test devices, you need to have included them in a specific profile through a web interface that’s not the best website I’ve seen out there, and you need to make sure to sign your application with this profile and none other. Problem number one: every time you need to add a new beta tester to the list, you need to go through the whole process of creating a new certificate, identical in everything to the previous one but for the list of authorized devices. Then, in the project settings, you need to make sure you’ve selected the new one (which has the same name as the old one, and doesn’t replace it). This is currently one of the major sources of my daily cursing spree (see below). Once all the beta testing has taken place, you’re home free, the rest is a lot easier.

One step more in that direction: working for a different company. They have their own developer identity, on which you might have only limited access. In that scenario, they have to manage the list of test devices, and grant you the right through a delegation mechanism to sign the application through their account. Basically, nothing changes, except the fact you can’t manage anything anymore: when they update the certificates, profiles and the rest… well if they don’t tell you, you’re screwed. Let’s assume for a second they gave you access and are extremely good at communicating that sort of information (that’s a big assumption).

When it’s not your own project, the pressure changes focus: you are not in charge of the timetable anymore. This means that any error that delays the time they get the application installed on their test devices can be a source for major conflicts. And, obviously, if they called in a contract developer, it generally means that they either don’t have the expertise or they don’t want to. Distribution of beta builds, when it could be done via email for your own beta testers becomes a little less diplomatic.

As many of my fellow developers, I have set up a server that distributes my work for me (kind of like the AppStore, in less pretty): On the test device, they click on a link, and it installs the application automatically. When you have one or two projects, you can do this all manually, it’s no big deal. Editing plists, HTML, and making sure the right profile has been selected is not that complicated. When you have ten projects and generate one or two builds for two of these projects each day, it becomes troublesome very fast. Mistakes are made, panic ensues, grief follows not far behind, hence the daily cursing spree (told you).

In Xcode 3, it was rather easy (when you know how, obviously) to automate that process: build the application, sign it with the right profile and certificate, push it onto the server, and add the relevant info to the web page. It can even optionally send an email to the right people for you. I fiddled around with my own build scripts, and with Hudson.

These build scripts (whatever the flavor) need a few things in order to work:

  • access to the code (obviously)
  • understand the structure of how Xcode generates the application (especially where in relation to the source code)
  • access to the provisioning profiles (more tricky if the server is not a Mac OS X machine, and that leaves a huge security hole as the process needs to access files in your home directory. Oh and to Xcode’s internal shenanigans, a profile doesn’t have a name, it has a fricking 60+ characters identifier, another source of potential mayhem)
  • access to your keychain (for signing, even bigger security holes as the process either need to be you or to have your personal password)

Fiddling around for a reasonable amount of time will get you set up. Lo and behold, when I commit something to my SVN, the server grabs the source, checks it, compiles it, signs the app, pushes it onto a web server, and sends an email to my customer. Perfect.

Xcode 4 on the other hand has changed all that: from outside of Xcode, you can’t know where the build has been put. It’s in the preferences of the application itself, it has nothing to do with the project anymore. So there’s an extra layer you need to manually add to your projects: putting the build result in a location that is always the same. Basically, you need to emulate Xcode 3’s behavior in Xcode 4. That’s typically a “new feature” that forgets most developers out there don’t work for themselves. And I read here and there that I’m not alone pleading for a return to a more project oriented structure: I have currently on my disk close to 4GB of builds that I know nothing about. Xcode 4 decides to put a bunch of things here and there, and if I trash the project, or revert to an earlier version, or change its name, or whatever project management thing we do on a weekly basis, these files will stay there till the end of times (or till I find the time to guess which is which and trash what I don’t need).

Wrapping Up, Because It’s Long Enough Already

I like a lot of things in Xcode 4 (workspaces, better source control management, UI designing features, and a lot of small features that made me smile the first time I discovered them). Some choices made are time consuming for me (why in hell can’t I lookup a symbol in the documentation without copy/paste???) but not that big of a deal.

To me, the only thing that is really troublesome is that most of the changes between 3 and 4 were made to keep you inside of Xcode. Good bye external scripting, good bye standard automation (recreating the same scheme over and over and over again with just 2 variables changed isn’t the same as having ONE script that adapts to the project), good bye workarounds for certificate/profiles management… And there is still no way to add plugins to integrate into Xcode what I need to do every day.

Xcode 4 is meant for the solo developer that will publish directly to the AppStore, with maybe a couple of beta tests thrown in. Every other type of developer out there will have to find a way to make it work for themselves, on their own. And we will, because we are, after all, in the same trade. That extra effort is not something I really care for, but I accept it.

Feel free to comment.