After 10 Years, Yelp Gave My App 4 Days

Update August 1, 2024

The original blog post below went a bit viral on Hacker News and garnered 60,000+ page views in just a couple days. Maybe that (or more likely other API partners complaining) had some impact at Yelp. They just sent the email that follows. I appreciate the apology and acknowledgement of the mistake. I also would like to clarify what I wrote in more words in the original post—Yelp is perfectly within their right to start charging for their API. They just did it in a very poor way providing us 4-days notice (1 business day really Friday->Monday).

Yelp's August 1st email (emphasis mine):

Earlier this month, we sent you an email about your Yelp Fusion API usage. That email gave developers until July 23rd to contact us if they want to continue using Yelp’s data for use in their app. We realize you might need more time and are extending your free access for an additional 90 days starting today. Your access should be available now.

We’re sorry for any inconvenience or frustration this abbreviated transition might have caused. Please respond to this email or contact us at api@yelp.com if you have any questions.

The Original Post

After 10 years, I have removed from sale my macOS app for figuring out where to eat, Restaurants. In this post I’ll explain what Restaurants was, how I developed it, and why it was removed from sale. It speaks to the challenges developers face when incorporating third-party APIs.

App Development Journey

The year was 2014 and Apple had recently released a beta of its new programming language, Swift. As an exercise for an early adopter of the language, I thought it would be fun to leverage my experience in a couple startups using the Yelp API to make a restaurant-search-specific Yelp client for the Mac.

I emailed Yelp, explained the app I intended to build, and got permission. The person in developer relations said since they didn’t have a native Mac client, they would approve my use of the Yelp API for Restaurants. In fact, without me specifically asking for it, they provided a 25,000 per day API call limit (a limit my app never even came close to reaching). It seemed they were in fact encouraging me to finish the app and release it.

This makes sense since Yelp limited severely how many results (20, later 50), reviews (3), and pictures (3) were available through their API. So, it’s not like my app could replicate the full experience of using the Yelp website or the Yelp iOS client. Instead, it served as a super fast, well-integrated with macOS, way to search for restaurants on your Mac with the option to go to the Yelp website to read more reviews and see more details. In other words, it was in many ways a traffic driver to the Yelp website.

I released the app and it sold moderately, but consistently. Over the decade it was available, I charged between $1.99 and $4.99 up-front for the app (a one-time purchase through the Mac App Store). Sometimes it was in the top 10 apps in the Lifestyle category on the Mac App Store. But it turns out you only need to sell a couple copies in a day to reach the top 10! The vast majority of sales were to US customers. I assume that’s either because the US is where Yelp is most popular or has to do with our restaurant culture, or the combination of the two.

I built the app using AppKit and some third-party Yelp libraries, including most recently CDYelpFusionKit. I was sure to integrate with as many macOS features as I could. For example, you could instantly call a restaurant on your phone using Continuity, add a Restaurant to your address book, or get directions in Apple Maps. You could share a Restaurant’s details with any of your social apps. And of course Restaurants could automatically detect your location for your search if you gave it permission. It was a small app, but it did the one thing it was meant to do, search for restaurants, well. I used it myself every time I thought about going out somewhere new to eat.

One of the most popular design decisions I made was to represent different categories of restaurants with different emojis. For example, traditional American restaurants would have the turkey emoji 🦃 next to their name. This allowed folks to quickly scan the search results for their cuisine of choice. You can see what this looked like in one of the screenshots on the Restaurants website. It was actually my wife’s idea. With her help, we figured out by hand what emojis to use for all of the many restaurant categories that Yelp provides.

Over the years as Yelp added more features to the Yelp API and rebranded it as Yelp Fusion, I added those features to Restaurants. In the most recent big update, version 3, I better-integrated Apple Maps for a more seamless view of your search results geographically.

Users loved the most recent version, Restaurants 3. Its reviews were all high. One reviewer wrote on the Mac App Store, “I’m new to the Mac world, but I just love this app. It’s beautifully designed, easy and flexible to use, and infiniitely helpful.”

The 4-Day Deadline

So, why did I remove Restaurants from sale? Well on July 19, 2024 I received the following email:

Hello API user,

We appreciate you signing up for and trialing the Yelp Fusion API.

Your API usage is higher than lots of other Yelp Fusion developers and we would like to learn more about how you’re integrating the Fusion API into your platform. Please share screenshots which represent how you implemented the Yelp Fusion API, display Yelp content, and clearly attribute Yelp to your users.

To continue using Yelp Fusion commercially, we need to complete a license agreement. We will send this to you directly after reviewing your product, use case, and Yelp Fusion Enterprise eligibility.

If we don’t hear back from you by 4:00pm EST on 7/23/2024, we will temporarily disable your API key until we receive a response with the above requested information.

Please disregard this email if you are already a Yelp data licensee. If you are a license holder, please send us a copy of your agreement so that we can ensure there is no disruption to your API key.

Thanks, The Yelp Fusion team

There are a lot of problems with this email. I’ll just name a few. First of all, I was not “trialing” the Yelp API. I had been using it for a decade and had official permission from Yelp to create Restaurants. Second, Yelp already had the information they requested. I sent it to them when I first created the app and requested API access. I also had a profile on the Yelp developer portal that described the app and they could easily access the Restaurants website.

Third, it was surprising to hear my usage was, “higher than lots of other Yelp Fusion developers…” That implies I am somehow abusing the system. In fact, it was Yelp themselves who (as mentioned earlier did it without my specific prompting) provided me a 25,000 per day API call limit. But the actual usage was on average below 100 API calls per day. As I mentioned, it was not that popular an app.

But the biggest problem with the email is the threat. It said that in 4-days they would shutdown my API key with no prior notice unless I responded with the information they requested. They sent the email on a Friday and wanted a response by Monday. Imagine I was on vacation? That would effectively shutdown all of my users.

I thought this must be some kind of mistake. I was not even aware that Yelp had gone to an exclusively paid-model for the Yelp Fusion API. I had not received any prior emails to that effect, and had not been keeping up with the Yelp Engineering Blog. So, the 4-day threat seemed completely out of the blue.

I responded to the email which was sent directly from a Yelp employee who I’ve chosen to remove the name of from this post and her email CCed another Yelp employee. So despite being written to “API User” it seemed like it was sent by hand and customized for each recipient.

At the time that I wrote my reply I still was not aware (based on the first email) that no free version of the Yelp Fusion API any longer existed since the email only mentioned “enterprise” and “data.” I should have checked the Yelp developer website to better understand the situation before replying but I was surprised and given the 4-day deadline wanted to get a response as quickly as possible given the threat. I wrote:

Hi -name of person who emailed me-,

This all feels a bit out of the blue with not a lot of warning. We have shared customers who have been using Restaurants for 10 years. The tone of the email feels pretty threatening after a 10 year partnership. We do not use Yelp Fusion beyond the limits that were provided to us by Yelp when the product first launched.

I asked for permission from Yelp before launching the product and received it. It mainly drives traffic to your website since you only provide three reviews and three photos per restaurant. Why would you want to shut down a product that simply serves as another way to drive traffic to Yelp? The reason you approved it 10 years ago is because you don’t make a native Mac app yourself.

The information about Restaurants you asked for can be found on its website: https://oaksnow.com/restaurants/

Please let me know what your intentions are going forward.

Thanks, David

Of course it was naive (or you could say hopeful) of me to think they would care about my little Mac app. Here’s the reply I got a couple hours later:

HI David,

Thanks for the response and information! We have converted to a paid licensing program. You will need to obtain a license with us in order to continue your usage of the API.

Without a license in place, your API key will be disabled next week. This will just disintegrate all Yelp data on your platform. You can always sign up for a free trial on our website if you need more time.

If you are interested in a license, please review the attached program and pricing overviews.

Thanks, -name of person who emailed me-

It was then I checked the website and found there was no longer a free API. I immediately removed the app from sale on the Mac App Store because I didn’t know yet if it would make financial sense to pay for the API. I didn’t want users buying the app and being shutdown next week.

Development Ends

I did email back and forth a little bit more just to say I wouldn’t have minded them going paid if I had received several months of notice, but that the 4-day notice and the threatening zero-context email was quite rude. I did this just to provide feedback since I knew the decision was fait accompli. Of course that didn’t go anywhere.

I started looking into my options. With the few sales Restaurants garners, it really didn’t make sense to start paying for the API. Especially with a company that would be willing to pull-the-rug from under me with 4-days notice. So, the app stayed off the Mac App Store. And today, 10-days after the original email, Yelp shutdown its API key.

This distresses me because there are users who purchased Restaurants (when it was on sale for $1.99 during an indie app sale) as recently as earlier this month and now it doesn’t work. Not to mention the customers who have been using it for years. But due to the way that the Mac App Store works we don’t have our customers’ email nor any way to directly refund them. So, instead I need to direct them to Apple to request a refund. If you purchased Restaurants, please ask Apple for a refund by following the directions here. Even though it was beyond my control and I took the app down as soon as I could, I sincerely regret your customer experience.

I’ve also considered moving to a different API but unfortunately no API is quite as comprehensive for restaurants as the Yelp API. Google Places is close. But it would also require a lot of re-development which is not really something I’m up for given the app’s low sales.

Lessons Learned

I have learned a lot from this experience. Perhaps the number one thing I re-learned, since it has bitten me before (hello Facebook’s Parse), is that if you utilize a third-party API for the core of your app, you are at their whim. But using APIs for many apps is inevitable. So, you need to be sure the companies you work with are not the kind that will give you 4-days notice.

The other thing this taught me is the danger of an up-front paid model for apps that depend on ongoing access to third-party services. If users were continually paying for the app, paying for Yelp’s APIs would not be as much of an issue. And I wouldn’t feel as guilty about the app being discontinued since if the fees were charged on a monthly basis, they would just end at the same time the app ceased to exist instead of facing an expectation upon purchase of “forever access.” On the other hand, how would you charge a monthly fee for an app that people are only willing to spend less than $5 for upfront?

Adding Contacts on Apple Platforms without Entitlements

Adding a contact to the user’s address book on Apple platforms using the prescribed method requires jumping through a lot of hoops. You have to:

• add NSContactsUsageDescription to your info.plist
• add an entitlement to your app
• ask the user for authorization (requires an in-app popup)
• if you want to modify the note field of the contact, you need to get permission from Apple using an online form to be able to add a special com.apple.developer.contacts.notes entitlement to your app; Apple may take up to two weeks to respond just to reject you

For one-off contact additions, there’s a simpler way that requires none of that. The idea is basic: you create a contact, save it in vCard format, and then ask the operating system to open the vCard file in the Contacts app. When the Contacts app opens, it will ask the user if they really want to add the contact. This requires no entitlements (even if you’re using the note field), no authorization, and even works in a sandboxed app.

The Code

I put a Swift macOS example app on GitHub. The code should be easily modifiable to work on iOS. Conveniently, the Contacts framework includes a method for serializing contacts into vCard format. You start by creating a CNMutableContact and filling it with your arbitrary data:

let contact = CNMutableContact()

contact.contactType = .organization
contact.organizationName = "Silly Restaurant"

contact.phoneNumbers = [CNLabeledValue(
        label:CNLabelPhoneNumberMain,
        value:CNPhoneNumber(stringValue:"5555555555"))]

let address = CNMutablePostalAddress()
address.street = "31 Silly Way"
address.city = "Silly Town"
address.state = "Vermont"
address.postalCode = "05401"
address.country = "USA"
contact.postalAddresses = [CNLabeledValue(label:CNLabelWork, value:address)]
contact.urlAddresses = [CNLabeledValue(label: "Apple", value: "https://www.apple.com/")]

Then you can serialize the contact using CNContactVCardSerialization and save it to a temporary file:

let data = try? CNContactVCardSerialization.data(with: [contact])
let fileURL = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent("contact.vcf")
try? data?.write(to: fileURL)

Finally, it’s just a matter of using NSWorkspace to open it in the Contacts app:

if let contactsURL = NSWorkspace.shared.urlForApplication(withBundleIdentifier: "com.apple.AddressBook") {
    NSWorkspace.shared.open([fileURL], withApplicationAt: contactsURL, configuration: NSWorkspace.OpenConfiguration()) {
        (app, error) in
        if let error = error {
            print("Had a problem trying to open the Contacts app.")
            // your error handling here!
        }
    }
}

I hardcoded com.apple.AddressBook as the app to open the vCard here, which is Apple’s Contacts app. You should probably not specify the exact address book app incase the user has a different default.

It works, but you might say, what about that note field? If you add a note property to your CNMutableContact you will notice it is silently dropped when the contact is added to the address book. This has nothing to do with the note special entitlement. It turns out CNContactVCardSerialization does not have support for either images or the note field. You can easily add both of these back. A Stack Overflow post provides some code showing how to do so.

Security Loophole?

When working on a new version of my macOS app Restaurants, I came across the note field entitlement requirement. I submitted a request to Apple using their online form to have access to the entitlement and a week later I was rejected for my request being too vague. Fair enough, it’s their sandbox, and they have the right to reject me for being too vague. But waiting so long to get an answer was frustrating and adding contacts requires a lot of ceremony. Frustrated, I went down the road of this alternative method for adding contacts.

It seems strange to me that you need authorization through Apple’s prescribed method just to add contacts but there is this “side way” using a vCard file that doesn’t require any authorization or entitlements. I’m not saying it’s a security vulnerability. It’s more of a loophole. If Apple really want developers to have to be authorized to do anything with Contacts, this shouldn’t exist.

I filed a security report with Apple thinking maybe they had looked over something so obvious. But I didn’t get a bounty. Instead, they closed it, and marked it as “expected behavior.” I basically expected as much. It seems like too obvious a thing to really be an oversight, but perhaps adding contacts should not require so many hoops to jump through using Apple’s way.

However, a key difference between the two ways of adding contacts, is that an authorized app has access to the contacts database, whereas this method using a temporary file simply opens the Contacts app, which is then in full control of the situation. It’s a lot safer. Any risk is mitigated by the Contacts app. But from a user perspective they look quite similar. In the authorization case, the user gets a pop-up within your app. In the example above, the user gets a pop-up within the Contacts app. The downside, is it does take the user out of your app. But if this is a rarely used feature of your app, maybe it’s worth it, to not have to deal with the frustration!

Book Review: Weaving the Web

Although it came out in the year 2000, Weaving the Web: The Original Design and Ultimate Destiny of the World Wide Web is as relevant today as it was when it was published. Weaving the Web is a memoir by Tim Berners-Lee about the creation and early growth of the Web. Most of the book concentrates on the ideas, insights, software, and previous attempts that led to the Web—as well as the decisions and evangelism that allowed it flourish in the ’90s. The last quarter of the book deals with Berners-Lee’s ideas about how the Web should evolve. Some of the philosophy behind the Web explained in Weaving is very relevant to current debates around censorship, centralized control of content, and privacy.

We cannot fully understand something if we do not understand its origin story. Weaving the Web fills in many blanks for the curious reader. Like any great creation, the Web did not form in a vacuum. It was the result of over a decade of experimenting by its creator. In the early chapters of Weaving the Web you feel like you are there with Berners-Lee and his colleague Robert Cailliau as they pushed the Web forward. Like Jony Ive has said—ideas are fragile when they’re first created. You get a strong sense of how Berners-Lee nurtured his idea.

Perhaps even more interesting than the origin story, is the philosophy and core ideas that Berners-Lee imbued the Web with. Some of his outlook is surprising and insightful. For example, he does not credit HTTP or HTML as the most important innovation, but instead the URI. “It is the most fundamental innovation of the Web, because it is the one specification that every Web program, client or server, anywhere uses when any link is followed.” (page 39)

Throughout the book, Berners-Lee advocates for decentralization and for empowering individuals as creators. It’s important to note that the book came out at the height of the Web 1.0 era, before the onslaught of social media and YouTube-like content sharing sites. A time when the Web was very static. Yet, he didn’t intend it that way. “I never intended HTML source code (the stuff with the angle brackets) to be seen by users. A browser/editor would let a user simply view or edit the language of a page of hypertext, as if he were using a word processor.” (page 42)

The first web browser that Berners-Lee developed was also an editor. He continually encouraged companies to come out with combined browsers/editors but most declined. It’s interesting to think how differently the Web would have evolved had the browser/editor concept taken off.

Berners-Lee’s philosophy goes beyond technology. He designed the Web to be an open, decentralized system that anyone could participate in. “Whether inspired by free-market desires or humanistic ideals, we all felt that control was the wrong perspective. I made it clear that I had designed the Web so there should be no centralized place where someone would have to ‘register’ a new server, or get approval of its contents. Anybody could build a server and put anything on it.” (page 99)

As the Web has become more and more dominated by a few large tech companies, many feel this early philosophy has been lost. It’s not the current ethos. It’s not the way that most people interact with the Web. Berners-Lee was very prescient in understanding this threat. “If a company claims to give access to the world of information, then presents a filtered view, the Web loses its credibility. That is why hardware, software, and transmission companies must remain unbiased toward content. I would like to keep the conduit separate from the content.” (page 132)

The last quarter of Weaving the Web deals with Berners-Lee’s vision for how the Web should evolve. Much of it did not come to pass—at least not in the way he advocated. It includes explanations of standards like SMIL that never really took off. It speaks to how creating a standard is not as important as making a killer app. This section is interesting from a historical standpoint—to understand what people were thinking about after the first decade of the Web. But it’s not nearly as interesting as the rest of the book.

Overall, Weaving the Web does a great job recounting the story of the Web’s creation. It’s well written and insightful. Most importantly, it clearly states the philosophical underpinnings that inspired Berners-Lee and propelled the Web through its critical first phase of growth. It provides a lot of historical context and insight for many of our current debates around the Web.

Introductory Programming Assessment Must Accommodate Copilot-like Assistants

GitHub Copilot and other machine learning based programming assistants will fundamentally change how we assess competency in introductory programming courses. Computer science educators can no longer rely on formulaic assignments and an honor code to ensure original work that demonstrates programmatic ability. Copilot-like assistants have blurred the line between the programmer’s work and machine generated code (largely modifications of pattern-matched work from their training sets). While they provide a productivity boost for the professional programmer and the advanced student, they may potentially act as a crutch for introductory students, who will rely on these tools in lieu of developing a strong understanding of their own.

There are certain standard problems that we are accustomed to assigning because completing them demonstrates the ability to implement fundamental simple algorithms. A prior strategy to reduce plagiarism has been to provide scaffolding code or put a spin on a problem to make it unique. Unfortunately, Copilot-like assistants are almost as capable in these scenarios as they are at writing generic simple algorithms. In my own preliminary testing on a (what I believe to be) unique scaffolded assignment of my own creation for an introductory class, Copilot was able to contextualize the comments and write most of the smaller functions accurately with just a little bit of my assistance.

What can we do about this? How can we assess competency in this environment? Over the past few decades, computer science education, like many other fields, has been moving away from exams and towards project-based learning. This has been a positive trend for a host of well-researched reasons that are spelled out in the literature. Unfortunately, I think this trend will need to be at least partially reversed for introductory courses. Students must demonstrate the ability to write and comprehend fundamental algorithmic code without the assistance of an AI. We could try banning it, but that never works well. Instead, we can try to assess knowledge “live.” How much do you know in this moment without someone or something’s assistance? And that’s what an exam evaluates.

Of course, exams have well-documented downsides, including but not limited to the fact that a significant number of bright students do poorly on them who do fine with project-based learning. I am not suggesting we return to a world of exams making up the majority of the grade in a course. Instead, I am suggesting that exams and exam-like evaluation will need to be a greater percentage of the mix. We can be creative. An oral presentation can in some instances demonstrate knowledge as well as an exam. A project coded live in class on machines that do not have AI assistants enabled, can serve as a pseudo exam.

We do not need to return to the dark ages. But we must acknowledge that these new tools mean that how we evaluate introductory programming knowledge has to change. I have had graduates obtaining first jobs in the insurance, defense, and healthcare industries building programs that have life-and-death consequences. They need to have a firm grasp of the fundamentals. What happens when Copilot makes a fundamental mistake and the person using it does not have the skill to realize?

My Experience with Technical Interviews in 2022

I just had the chance to experience 8 different technical interview processes. And they were not what I expected! I didn’t get asked a single data structures and algorithms question. No leetcode for me. This may largely be related to the types of organizations that I applied. Despite writing a book about algorithms and teaching a class called “Data Structures & Algorithms,” I don’t do particularly well at white-boarding. I hadn’t done a software development job search in 10 years, and based on what I had read, and what I had heard from my students, I expected to do some white-boarding. But I didn’t have to do any white-boarding!

I wanted to write this post as a kind of follow-up to a podcast episode we did a few months ago about technical interviews. We really emphasized the prevalence of algorithmic problem solving questions and brain teasers. Maybe we got that wrong. Maybe that’s an outdated view. Or a view limited to only certain kinds of companies.

In this post, I’ll tell you about what the processes were like at the anonymized companies that I applied, how the rounds were ordered, and how the search ended for me. But first some background for context: Earlier in my career I worked as a full-stack web developer at a couple startups and then worked for a few years as a freelance iOS developer. Then I shifted to computer science education. The past six years I’ve been working as an assistant professor of CS at a teaching college. In the past three years I’ve been the co-program director of the CS program.

So, I’m a strange candidate. I didn’t know if I should be applying for mid-level roles or senior-roles. I don’t have any corporate software development experience. I do some open source work and write books and teach. I have a few very small indie apps. And as I mentioned, I worked for a few years in startups and as a freelancer. So, it was hard to tell how companies would see me.

I did a mini-job search this month. I had the option of a promotion and a new contract at the teaching college I work at, but I wanted to see what else was out there before I signed the dotted line. I want to emphasize that I was sincerely interested in exploring changing careers back into the world of software development. I was a serious candidate. This wasn’t a lark. At the same time, I knew there was a reasonably high probability I would choose to stay in my current profession.

That’s why I decided to do a very targeted search. I only applied to a select group of (mostly remote) roles that seemed like a very good fit. And I did all of the applications in 1 day. After that day, I didn’t apply to any more companies. The entire process from doing applications to offer was about 2 weeks.

How it Started

On May 2nd, I applied to a little more than a dozen jobs on LinkedIn and via the iOS Dev Jobs mailing list. I also posted on the monthly Hacker News “Who wants to be hired?” thread. That was it. I never applied to any more jobs after May 2nd.

By Wednesday I had received a few immediate rejections, 5 first-round interviews as a result of my applications, and 3 inbound first-round interviews from companies that saw me on Hacker News.

How it Was Going

I got a second round with just about every company I applied to. Apparently I did very well in those first round interviews. By the end of the first week I was very confident.

How it Ended

I’m going to break your suspense before I get into the details because it will help frame some of my decisions around why I pulled out of several of the companies’ processes. I received two offers by May 17th. They were time sensitive. I ended up accepting an improved offer to stay at my current employer. The other offer I received was for a senior software engineer position with a salary that is literally more than double my current salary. It was very compelling and I really liked the person I would’ve been working for and his team, but I ultimately decided I didn’t want to give up the academic lifestyle. I really do like where I work. It’s a great place. But it was a hard decision. I also received two outright rejections from companies that I had gone through three rounds with. I pulled out of the other interview processes after accepting the offer to stay.

The Processes

I am anonymizing all of the companies here as much as possible. I’m not here to criticize anyone specifically, but instead praise and criticize some of their interview processes more generally.

Company 1 — to be a Mac software developer

Company 1 is a small/medium sized maker of software for the Mac. I make indie Mac apps myself and first learned to program for the Mac when I was 14 years old. So, this was a very interesting company to me. And something I knew I would enjoy doing.

Application or Reach Out

I applied.

The Process

The first round was a 30 minute screen with the co-founders of the company. The second round was a technical interview that was just a conversation about some of my past work and programming philosophy with the CTO. The third round was a take-home software development project that you had unlimited time to complete and were paid to do.

Result

I pulled out before completing the third round because I had received the other time sensitive offers. I think I really would’ve enjoyed working for them, so I asked the CTO to keep me in mind for part-time or contract roles in the future.

My Feelings

This was a very good process for a small to medium sized company. It was very personable (meeting with the owners right off the bat) and yet also seemed to hit at the technical in exactly the right way. The technical conversation was non-intimidating and I think really did give the CTO a good sense of me and what my skills are. The take-home project was very relevant to the work they do and a good test, although I pulled out before completing it.

Company 2 - senior software engineer working in Python

Company 2 is a Fortune 200 company looking for an addition to a relatively small inner team that works on B2B products for an exciting and growing market in green energy.

Application or Reach Out

The hiring manager reached out to me based on my Hacker News post.

The Process

The first round was a 1 hour conversation with the hiring manager. I appreciated that he took the time to talk to me for that long instead of sending me to HR. He was erudite and personable. He was specifically looking for someone who could be an internal teacher which I really appreciated and made the job even more compelling to me. There was a short call with HR after. The real second round was a 5 hour interview with 5 inner rounds: 3 technical interviews (data modeling in Python, testing/QA in Python, and commenting on a Python pull request), a culture interview with 2 engineers, and a final interview with the hiring manager.

Result

I received a very compelling offer for more than double my current salary. I did seriously consider taking it, but ultimately accepted the improved offer from my current employer for the reasons I mentioned above.

My Feelings

The hiring manager was fantastic and I really enjoyed talking with him. If he hadn’t been so great, I could see a one hour first round being a bit much. On the other hand, the entire interview process really only had two rounds, so the entire process is very reasonable. While I don’t love live coding, the engineers on the live coding technical interview rounds really made me feel like it was a collaborative process. They definitely weren’t checking for syntax. They were checking for thinking. They helped me when I got stuck without giving too much away. The pull request technical interview was great because it really let you shine as a developer in a different way—as someone with judgement instead of just someone who can code. Overall it was one of the best renditions of live coding I’ve seen.

Company 3 - iOS developer at a consulting company

Company 3 is a reasonably well known consulting company. I used to work as a solo iOS consultant (freelancer) so this role seemed particularly compelling to me.

Application or Reach Out

I applied.

The Process

The first round was a 1 hour conversation with a senior engineer there who seemed to be guiding the process. The second round was a 1 hour technical interview with another engineer there that asked me technical questions about Swift (i.e. how do closures work in Swift). The third round was first some simple live coding in Swift, and then building a very simple table view based app in Xcode. I believe I completed all of the tasks successfully. The two technical rounds both ended with some more “soft questions.” In the third round I was asked by one of them if I had any “enterprise development experience.” And I said I didn’t have any, which is true and was evident from my resume.

Result

I found out just after I had received the two offers mentioned above that I was not selected to continue to the fourth and presumably final round. I had actually already accepted one of the offers, so I was about to pull out of the process anyway, but they emailed me first, so I unfortunately got to feel the rejection! They said the reason was that I didn’t have enough enterprise software development experience. Which makes sense because I don’t have any. But they knew that before the first round, so it was a strange reason to reject me. Perhaps they just had a better candidate who did, or perhaps they didn’t want to tell me the real reason.

My Feelings

The first round was great. They said they were interested in my book writing and podcasting work. I found the second round interviewer hard to read, but the questions he asked about Swift I thought were fair. I heard after by email that I did well. The third round was not great. It was live coding doing some fairly trivial stuff, but the lag of using a remote system did not work well for laying things out in Interface Builder. I did have trouble remembering during the live session how to do the syntax of some simple exception handling in Swift, so maybe that’s the real reason I failed that they didn’t want to tell me. I did get it right eventually, but it took me longer than it should have to remember the syntax. However, I don’t think asking live syntax questions is a good way of conducting technical interviews—in the real world we can just look it up. And I did politely criticize the laggy setup at the end too by suggesting it would’ve made more sense to let me share my screen instead of remotely controlling theirs (probably not a good move to criticize your interview setup). Ultimately the feedback for why I didn’t get to the final round felt too short—I’d like more detail. And if the “not having enterprise software development experience” was the real reason they should have screened me out after the first round, not the third. I think doing a live coding session in Xcode makes little sense in the iOS world—let us do a take home or look at our prior open source work or indie apps. That’s a better test of what we can do. However, I sincerely liked the company and could see myself applying again if I ever do another job search.

Company 4 - senior iOS developer at well known iOS app company

Company 4 is very well known. Their iOS app has millions of users. They were looking for someone to be in a senior role on the iOS team. They described it as a leadership role in the interviews.

Application or Reach Out

I applied.

The Process

The first round was a 30 minute screener with HR. The second round was a 30 minute technical conversation with the head of the iOS team. For the third round you were offered either a 2 hour live Swift coding test or a take home. I chose the take home. It was building a fairly simple iOS app but with plenty of room for embellishments. They said you had as much time as you wanted, but also said not to spend more than a few hours on it. After two days I got a message from HR saying they were eagerly awaiting my submission (I hadn’t started because of other interviews and them saying I had as much time as I needed). I told them that I had been busy (the truth) and would finish it by the end of the weekend (4 days later, which I did). I sincerely thought I did a fine job on the app. I even added a couple extra features, like some custom bar chart drawing, that was not required in the app description.

Result

I was not selected after the third round. The email from HR said “Thank you for taking time out of your schedule to complete our assignment. We appreciate the time and care that went into completing the assignment. At this time, we have received several assignments back that have set our candidate caliber very high. It has not been an easy decision but we will not be advancing to the next round interview. Please continue to view our job board for any future openings you are interested in.”

I replied because I really wanted more specifics after doing the take home because I thought it was reasonably good: “Thank you. I understand and appreciate the consideration. If I could receive any feedback about what I could have done better on the assignment I would appreciate it so I can improve my work in the future.”

The final reply from HR was: “The feedback was all positive, we just didn’t have a good match on the technical side. I do wish you the very best.” I’m not sure exactly what that means. Was the feedback all positive on the technical side but the techniques I used were not a good match? Was my code amateurish in style but good logically? Was my use of older technologies not good? I really would’ve liked more feedback after the time spent on the take home.

My Feelings

I felt this company had a very fair process. But their final feedback was not great at all. After spending several hours on an unpaid take home project, you’d like to at least know the specifics of why it wasn’t good enough or why overall you weren’t a good candidate. That would have to come from engineering, not HR of course. But HR could pass it on. The most likely explanation is that they were telling the truth in the first email—there simply were higher caliber candidates applying at the same time. But the second email then made me doubt that. What does it mean for the feedback to be all positive but to not have a “good match on the technical side.” What is it that I could’ve done differently to bring my take home to the next level and be a good technical match? I sincerely wanted to know. I thought my app was pretty good for the time they told us we were expected to put into it (“a few hours”) and not getting to the final round honestly gave me a bit of imposter syndrome :).

Company 5 - educational non-profit lead developer position

This is a non-profit with a strong engineering culture. They were looking for someone to lead one of their large initiatives which is based on a Web app written in Ruby on Rails.

Application or Reach Out

I applied.

The Process

I first had a 15 minute chat with the founder. He was super nice. The real first round though was a 1.5 hour conversation with him. It was wide ranging. We talked about technical stuff including him asking me to show him some of my work, and he asked me some well prepared interview questions about product improvements they could make and how I saw the product. The third round would’ve been working directly on their codebase (paid) to implement a couple features after speaking with two of their engineers for about an hour. But I pulled out before the third round. One thing I really didn’t like was that in preparation for the third round you had to sign up for all of their services (Slack, Stripe, Logging service, etc.). In total I received 7 invite emails for the third round. It was a bit much.

Result

I pulled out of this one before even receiving the other offers because I had a bad feeling before the third round. I decided I really didn’t want to work in Ruby on Rails (bad memories of a startup I did 10 years ago) and I also had a feeling that I wasn’t a good culture fit. I felt a bit bad pulling out the night before the third round but I didn’t want to waste their time or my own.

My Feelings

It’s a super cool non-profit but the third round process didn’t make a lot of sense. Sure, you want someone to show they can work on your actual codebase. But requiring them to sign up for 7 different services is a bit much when they are likely doing other interviews. I’m glad I pulled out because I did a lot of hard thinking about it and really knew it wasn’t a good fit for me technically or culturally. It was a tough decision though because they really do have a great mission and great founder. But I think this made me realize that the technology and culture I’m going to be working in is as important as the product.

Company 6 - developer relations at advanced startup

This is an exciting startup that’s pretty far along. They have a product that is solving real problems for real clients already. Developer relations seemed like a good intersection for me between my technical communications skills (teaching, writing, podcasting) and software development abilities. In some ways it seemed ideal, but at the same time I didn’t have any background in working with their type of product.

Application or Reach Out

They reached out to me based on my Hacker News post.

The Process

The first round was a 30 minute screen with one of the founders. The second round was a 45 minute conversation with their head of engineering. The third round would’ve been a written interview (you fill out 5 detailed question in a Google Doc).

Result

I pulled out after the second round because I received the other offers and had to decide between them. The timeline just didn’t work out. But they actually reached out to me after I pulled out to ask if I would reconsider. That was super unexpected and flattering. I was very impressed by them overall.

My Feelings

It was clear in both interviews that the team is experienced and intelligent. I have regrets about not getting to work with such a sharp team in a role (developer relations) that I think really exploits all of my different skillsets. At the same time, I was a bit hesitant to work in developer relations on a product that’s in a space I didn’t know anything about. I have a strong feeling they will be very successful and kind’ve have some FOMO about it. I also regret not getting to do the written interview process because I pulled out. I’ve never done a written interview before and it seems like a very unbiased way to judge a candidate.

Company 7 - iOS lead at small startup

This is a startup in a space that I would call a form of social networking. They were well funded, but funded by friends & family, which is a bit of a red flag for me based on my earlier career.

Application or Reach Out

They reached out to me based on my Hacker News post.

The Process

The first round was a relatively casual chat for over an hour with one of the founders. It was really wide ranging and I felt like we really connected. The next round was supposed to be a conversation with the more technical co-founder, but it never happened before I had to pull out due to the other offers.

Result

I pulled out as mentioned above because of the other offers.

My Feelings

This company is early enough and small enough that they didn’t have a great technical screening process in place. I actually mentioned that to the founder and maybe it’s something I’ll even help him with in the future. We’re still in touch by email and the product is definitely interesting. At the same time, I would’ve had some real hesitancy to join a startup funded by friends and family because I did that a decade ago and we ran out of funding.

Company 8 - software developer at huge multi-national

This was a poorly defined role at a subdivision of one of the most well known technology companies of the 20th century. The job is at an office near where I live, so I figured it was worth a shot to at least find out more.

Application or Reach Out

I applied.

The Process

The HR representative reached out to me by email. I replied. She didn’t. She reached out to me again by phone. I replied. She didn’t. We finally connected for the first round, which was an HR screen.

Result

I pulled out during the HR screen. The salary was too low.

My Feelings

This company is out of touch and it showed even in the application process. The UI of their job site was the worst I’d seen. Also, companies should post salaries on job descriptions so as not to waste people’s time.

Remainder

I got rejected without even an HR screen from about seven other companies that I applied. They were mostly big companies. This kind of makes sense because my resume is very non-traditional for these kind of roles and I doubt someone who is looking at me as the whole person and potential developer (for example looking at my open source projects) is necessarily working in HR at a big company. Smaller companies seem to more often have people with software development experience doing first rounds.

Conclusion

Overall, I got a great offer in just two weeks. I can’t really complain about that. And interestingly it was from someone who reached out to me on Hacker News instead of somewhere that I applied. While I ultimately just accepted an improved offer at my current employer, I came away from this process with three takeaways:

1. If you have a non-traditional, but strong background, having that be well understood by a hiring manager is really important. You need to put all of yourself out there and get that information to that hiring manager in as succinct a form as possible. My Hacker News post did that for me.
2. A lack of feedback after doing a take home test doesn’t leave a candidate feeling good. Employers, please respect that people put time into doing a take home and give them detailed feedback.
3. leetcode problems do not seem as prevalent as the zeitgeist would have you think. Or maybe that’s the selection bias of where I got interviews. I mostly didn’t interview at very large corporations.

Building Retro Dither: Dithering Algorithms, Run-Length Encoding, Resource Forks and More!

Today I launched a new novelty Mac app, Retro Dither. Retro Dither gives any photo a cool retro look using just black and white pixels. You may want this for artistic effect, or you may want to export your photo to MacPaint for display on a retro Mac. Retro Dither launched on the Mac App Store today.

Note that we’re talking about just black and white, not grayscale (no grays here!). Amazingly, dithering algorithms make pure black and white images look like grayscale. You’ll get an old-school newspaper look, or a look like your photo just came off of a Mac from the 1980s. And this works with any photo in any bitmap format (JPEG, PNG, HEIF, etc.). Retro Dither can then export your dithered image to MacPaint format for display on black and white Macs going all the way back to the 1986 Mac Plus.

This is the story of how I developed Retro Dither…

I was working on my next programming book, which will be an intermediate Python projects book, when I came across an article about Atkinson Dithering on Hacker News by John Earnest. Dithering algorithms can be used for approximating the look of an image with less colors. Atkinson Dithering is one that is particularly well suited for approximating an image using just black and white. I was in the middle of developing a project for the book related to computational art and I was thinking I would need another art project to fill out the chapter. And Atkinson Dithering really caught my eye. First, because it looks cool. Second, because I was a Mac user growing up and I remember all the software and games that actually used it.

But there was a problem. After reading John’s article, I found the algorithm too simple to sit alongside the other projects I was developing for the book. I can explain the gist of it to you in a few sentences. You take each pixel in an image one at a time from the top left of the image to the bottom right. You check if it’s closer to black or white. Whichever it’s closer to, you turn it into (if we stopped there, we’d have a simpler dithering algorithm known as Threshold and it wouldn’t look great). Then you take the difference between black or white and the original pixel and call that the “error.” You spread that error into some of the pixels to the right and below the original pixel. Then you continue to the next pixel. John explains it in more detail in his article.

Then I got an idea. Bill Atkinson, the creator of Atkinson Dithering, was also the creator of MacPaint, a program I loved growing up. What if we not only dithered the image, but then we exported it to MacPaint format? That would be really cool and would add a bit more heft to the project. So, I started researching the MacPaint file format and found this article. It’s not a particularly complicated file format, but it at least includes another useful algorithm, Run-Length Encoding, a simple compression scheme where instead of writing out repeated sequences like “AAAAAAAAA” we essentially would write 2 bytes that say “9 As”. Now we have two simple, but interesting algorithms. That’s perfect for a project in the book.

I had to learn a bit more about the MacPaint file format. For example, it encodes pixels at 1 pixel per bit before it does the run-length encoding on the pixel-packed bytes. 0 is for white pixels and 1 is for black pixels. It runs the run-length encoding on a per-scanline basis. Every MacPaint file contains exactly 720 scanlines and 576 pixels per scanline (basically a 576 x 720 resolution). I found my hex editor, Hex Fiend, really helpful when debugging. I would look at my output MacPaint files and compare them to example MacPaint files, including some I still have saved from my childhood. I also found an open source project that converts files the other way (MacPaint -> modern bitmap). I could run my program’s converted MacPaint files back through it and see if the same image came out the other end.

With the help of Pillow for reading the initial bitmap image for conversion, I was able to build the whole thing in a couple hundred lines of Python. Again, this was the perfect size project for the book, so I was feeling really good about it. Then I tried my MacPaint files in actual MacPaint and there was a problem. They wouldn’t open!

Thinking back to my early days in computing, I remembered about type and creator codes. These were metadata that the Classic Mac OS (versions 1 through 9) used to associate a file with the program that should open it instead of using a file extension. The problem was my files had none. I used a Classic Mac program to add them back in and my images successfully displayed in MacPaint! But this wasn’t an ideal workflow. I wanted my users to be able to just double-click their finished files. So, I wanted to add the type and creator codes into the exported MacPaint files from my Python program.

The problem was, the Classic Mac OS stored files in two separate sections, known as data forks and resource forks. It stored the type and creator codes in the resource fork. Almost every other operating system, including modern macOS, Linux, and Windows doesn’t have resource forks. Files are just a single blob. And my research was showing me that there would be no easy way to add a resource fork to my file for transport so that the Classic Mac OS would see it as one file. I needed a work around.

Of course tons of people had the same problem before. Classic Mac OS users had to work with people on other operating systems. So transport file formats were developed that could combine data forks with resource forks together in one file for transport and then “unbundle” them on Classic Mac OS so they could be used. The simplest and most common (other than the later developed .sit, Stuffit archives, basically Zip files in Mac land), was MacBinary.

Turning my MacPaint files into MacBinary files for transport with correct type and creator codes was as simple as adding a 128 byte header to the file in MacBinary’s specified format. This is still not a perfect solution, because the files need to be unstuffed by a program on the Classic Mac OS side that understands MacBinary before the MacPaint files can be double-clicked, but programs that understand MacBinary were common on the Classic Mac OS and include the aforementioned and ubiquitous Stuffit.

I got this all working in a couple nights for the book and then I thought, surely someone else must make a commercial MacPaint converter. The retro Mac community is pretty huge right now. But I couldn’t find any, other than the venerable Graphic Converter, which is a full-featured graphics program that I recommend (its creator, Thorsten Lemke, was actually gracious enough to let me conduct an interview with him for a research project in college back in 2006) but that costs $30.

I decided to make something lighter weight. Thus, Retro Dither was born. I ported my Python code to Swift. I added 4 more dithering algorithms (they’re all similar to implement). I built an AppKit frontend. I tested it. I tested the output files in emulators like SheepShaver for MacPaint 2 on Mac OS 9 and Mini vMac for MacPaint 1.5 on System 6. I did find a couple bugs, like that MacBinary expects each fork to be in exactly 128 byte chunks (I just had to fill extra space with some 0s). Even though later versions of Stuffit will ignore this, other MacBinary unbundlers like the built-in MacBinary command-line app on modern macOS and the original MacBinary program from the the standards authors in the 1980s will not (by the way, the modern macOS’s Archive Utility will unbundle MacBinary files).

I hope you enjoy Retro Dither. And if you want to be made aware of my next Python book when it comes out, which will include the project described here, please join my very low volume mailing list.

Apple Apologetics

The common criticism of the modern news industry is that objective journalism has been replaced by opinion-slanted proselytizing. There’s an interesting slice of the news world where proselytizing was never even at issue. I’m talking about the world of news sites, blogs, magazines, and rumor mills that cover Apple.

I’ve been reading news sites like MacNN (now defunct), MacWorld, and MacRumors daily for more than two decades. I also enjoy more personal “news” and commentary blogs like Daring Fireball and Six Colors. And then there’s the journalists like Mark Gurman and David Pogue that at some point in their careers covered tech in an Apple-centric way for real news organizations like Bloomberg and the New York Times. There’s a cottage industry of journalists and outlets that make their entire living on the happenings of Apple Inc. Surprisingly, many of the specialized sites even managed to scrape by when Apple was merely a ten billion dollar company, and not the two trillion dollar behemoth it is today.

There’s an important assumption about the readers of these outlets. It’s assumed you’re a fan of the company. Why would you read MacWorld or a lot of what David Pogue wrote back in the heyday of his Apple fervor if you didn’t own an Apple product? But being a fan of Apple generally means more than being a user. In the 1990s when, as a kid, I first became a rabid Apple fan, it was known as a cult (in fact one of the news sites is still known as Cult of Mac). Why were you using the non-standard products of this small struggling computer company with less than 5% marketshare? You had to believe in their products’ superiority or the potential of the company’s future despite the clear market signals to the contrary. You had to believe. It was a religion for techies.

That belief in Apple and its products has always been imbued in the outlets that cover the company. The fans demanded it. It was why we kept reading. We wanted our faith to be reinforced despite all of the contrary voices. Reading the news sites was like going to church. It made sense in the context of the ’90s. They were the underdog. It was fun and cool to root for the rebels. And the potential was there. If we could just keep them going with our collective enthusiasm, they could reach their true destiny.

But then they did. They became the largest company in the world. They came to dominate several slices of the tech industry. You’d think that would mean the coverage of them would change. Yet, the cottage industry that covered them never lost its fervor. It’s like the prophecy was fulfilled and now those who believed would go to heaven and be angels, rewarded for their faith.

And angels they are. Towards the company. The new presumption is no longer that some unrealized dream could be achieved when the right products arrive. The new presumption is that the company is a force for good. How did this transformation happen? It used to be about “could they deliver the long promised tech future.” It was about Mac vs Windows. For a while it was about iOS vs Android. Now it’s about social issues, battles with government, and court cases.

I think that we’re all looking for something to believe in. And many of us, including the “journalists” and commentators that cover Apple, are not finding it in the rest of our lives. Yet somehow our belief in this company during its dark times, against all odds, was right. And so we want to continue to believe our faith in it will be right again. Yet, today, we’re being asked to believe in something very different. A different company facing a different set of challenges in completely different spheres. I’m still a big fan, but I no longer take every leap of faith.