da Bomb - Badge Post-Mortem

We're done! Learn from our mistakes!...

3 months ago

Latest Post da Bomb - Badge Post-Mortem by John Adams

Intro

Welcome to our second badge post-mortem. John here, Team Lead, Team Ides.

We’re going to talk today about we did right, everything we did wrong, and how we think things went. In general, “da bomb” was one of the most successful projects we’ve ever done and we’re proud to say that our planning worked.

There’s a bomb-shaped hole in our hearts now that the project is done, and we’re eagerly working towards next year. This document is a way of capturing what we learned and is a work in progress and we will update it as we remember our triumphs and failures.

So, how’d it go?

Planning


First off, we had a huge head start on the build because we hadn’t done a badge for DC26. Personal issues and other problems got in the way.

In that time, Bill had done an enormous amount of work porting ChibiOS to the BMD340, saving us from that issue this year.

I feel like overall, planning was very good. The schedule we posted during the Kickstarter Process was fairly spot-on, give or take a few weeks.  We paced ourselves against the DC25 schedule we’d previously accomplished and while at some points the DC27 build lagged behind the DC25 work, we were producing 500 vs. 225 badges, and eventually we went faster than the DC25 build.

We budgeted about two months of slip into the schedule and used up nearly all of that time! The major problems contributing to slips in our schedule were:

Income and Expenses

Most of our money went to MacroFab.


The total badge cost was $45,433.98, not including rooms, transportation, and defcon badges for the team.  All-In, costs get a bit closer to $53,117.16.

We raised $39,426.55, putting us over budget by $13,750.61 (fuck.).

I had to float that on credit cards, and that sort of sucked. Another $1,100 off to Sixt Rental agency for a Jeep Wrangler for the week and we were off to DEF CON.  Sixt was very easy to work with and I’ll use them again.

A single board's BOM (Bill-Of-Materials) clocked in around $93. When doing the pricing, I forgot to include ancillary expenses like prototyping, bank fees, and shipping from our vendors. The mistake we made here is that we were paying an additional $15.37 per badge, bringing the "profit" margin to around $11 badge. Margins were really that slim.

Pricing

We sold the badges between $120 and $150 depending on where you bought and how you paid. Pre at $120, Cash at about $130, Stripe/Square sales at $140 to cover the 2.5%+0.25 fee.

Interestingly enough, people at DEF CON were willing to swipe/dip their credit cards into my square reader. The trust is that strong in the Square brand now I guess.

At times, some revenue had to go to to vendors and other middle men like credit card fees. 30% is a pretty universal margin on most things sold in the US, and we feel that we were in a good price range (but should have gone higher – the market would have easily bared higher prices given the limited availability of our badges and functionality.

We were probably the 1st or 2nd highest priced badge but we feel that we offered a lot of art and functions, so it was okay.

After moving so much money around to multiple continents and vendors,  we maybe, just maybe squeaked out a profit of about $195.09 at my best estimates. That's #badgelife! Given that there's badges left, and we had to ship out a replacement or two, we're technically still in the hole. :(

Like last time we bought a tiny bit of equipment to make things go faster. A new Hakko FR-931 soldering iron was acquired and heavily used.  

Back in DC25 we bought a new Oscilloscope, AmScope microscope, and Hakko hot air station. Those purchases saved the entire project.

We used Square for in-person sales (the $32 reader was a good value), and Stripe for online sales with custom code. It worked out very well.  I only wish I could have done all sales on one merchant account. It’s not easy.

Hacker Warehouse was also a huge help, selling around 100 of our boards in the vendor area. So, a special thanks go to ggee for putting up with us, and another big thank you to our sponsor, New Context, we couldn’t have done this without them.

Limited failures and an excellent yield of 99.4%!

Yield was incredibly good this year, so good we couldn’t believe it. Out of 500 boards, 496 boards were functional with the remaining boards having only minor issues.

Overall the build and packing out of Macrofab was very good. Macrofab has come a long, long way and our design skills have improved in the last few years. Did I mention I finally built a completely silent audio circuit in stereo? Yeah!

Hardware mistakes

Software


Some stats:

Let's have a massive round of applause for Bill. Bill kicked serious ass at debugging, firmware, drivers, and all things in general. Couldn’t have done this without him!

I have to say that throughout the project the one thing I really didn’t want to face was the game. It was going to be very hard to write (real-time, multiplayer code isn’t easy), involve a technology I hadn’t worked with before (Bluetooth Low Energy) and a multitude of new game problems. Once I started coding it, it seemed easy, but the sprite library and inability to have enough memory to double-buffer the screen made for some fun times. Don't be like me. Get a MCPU with enough RAM.  But hey, on to the game...

Game Design

We're pretty married to uGFX on our badges. We know it well, and can make it do things that it probably shouldn't be able to do like sprites and overlay.

The game was created by taking the state machine from the DC25 Roman Fighting game that we made and then designing around that engine. I wrote a very basic rigid physics engine (all in C) that took into account things like drag,  acceleration, ship size, and motion. A bit of code later and we had the world map done. Later, we threw out my early code base and replaced it with Devon's sprite engine.

Once I had a basic motion system, Egan drew a map and ships, we wrote code to generate bullets, collisions, and motion. We wrote all of this by hand in C with very little in the way of external libraries or support aside from uGFX. It was grueling.

I really want to be able to use PyGame or something else on our next badge and worry more about the game and less about the game's engine.

It took weeks to get a game that Egan could come by and tune. That is, we had to have something playable before we could work on the rules, and we didn't have a lot of time to refine them. Egan and I had quite a few late nights tuning the game, and Devon came in a few times to assist us with the sprite library he wrote.  It solved a lot of problems for us like working in limited memory and partial updating of graphics, which made things run a lot smoother.

The game is fairly fluid, but we did ship with some bugs, so let's talk about that...

Software Mistakes

There’s still a couple of unfortunate sprite glitches in the game. If you go to the far right on the screen, it will incorrectly draw the ship, and I haven’t had time to find this dumb bug. We’ll get to it eventually!

Everything else worked fine, and the extra Flash+RAM on the BMD340 made life real easy.

Did you know? There’s nearly 4GB left on the SD card and 600KB of flash for you to play in. Reformat the SD card. We shipped a 4GB partition over to our copying company, and all they had left was 8GB SD Cards, so you got 4GB for free, matey! Go write a game or something!

We had a last minute, show stopping network bug -- One thing I did not understand is how the Nordic soft device actually sent data. You put data into a buffer in memory, tell the soft device to send it, and then you wait until you get an acknowledgement event. This all happens asynchronously.

My original software was loading data into the same buffer and changing the buffer over and over again. This would cause the packets to be clobbered, and the game to end. Bill and I worked out a last minute ring-buffer fix that made the game go smoothly.

We’d still like to find a CPU with enough memory that we can safely double-buffer a large screen. Our badge for next year might use that, if we can find something we like.

Shipping and Pickups, or, why you shouldn’t do pickups.

Shipping is surprisingly expensive. Around $1,300 of the project went to shipping. We charged people $15, but that was including packing materials, box, and USPS, so let's say we made maybe $1 on shipping there.

We kitted and shipped 160 badges, two weeks before DEF CON by USPS Priority mail. Average shipping cost was around $7.95, and we shocked the USPS lady when we walked in with a massive cart of priority mail!

All orders made it to their destination with the exception of one which was lost in the mail (USPS said it was signed for and picked up, we’re still filing a claim for our $100 or so)

We then did three pickup windows at DEF CON. I never want to do pickups again.

They are too confusing, and when 100 people descend on your hotel room, and you’ve got a crew of six people kitting, testing, and putting badges in people’s hands right then and there it gets pretty hectic.

A couple of times we marked I accidentally marked an order as shipped when it wasn’t because of issues with Shipstation and poor network response in Bally’s (thanks DEFCON!). This required some apologizing, but eventually we sorted all of the mismarked orders out.

ProTip: It's incredibly important to have a paper copy of your shipping / pickup manifest when you are at DEF CON. The network might go away. We found that practically no one lied about their badge pickup status, but we had a hard time keeping track. We double-tracked orders on shipstation and on paper.

Currently, we're still sorting out about 15 or so shipments which people haven't picked up or called about after we sent multiple emails! arrgh!

We also spent two 8-hour shifts in the HHV waiting for people to show up. Rarely did they show up after day one, which took away a lot of valuable DEF CON time!

Shipping really is the way to go. You can resolve all of the issues before the con at your leisure. This also means keeping at least three weeks between your ship date and the event, or you won’t have time to ship and fix things.

We created SKU’s for each product type -- either you got the challenge coin and the sticker sheet and some extra goodies, or you did not -- and then started to import data.

We were extremely reliant on shipstation.com, which is part of stamps.com. We were able to import backer reports from Kickstarter, orders from Stripe, and manual orders with simple python scripts. Those CSV files were then used with shipstation’s label maker to create single-sheet packing list and shipping label sheets which we could use to ship our orders. Combined with the free bags and obscene amounts of bubble wrap from Macrofab’s shipment, we were able to ship easily.

Things for Next time

I’d like to only ship. No pickups at all. I feel like I'd be much saner.

I’d like to not use Kickstarter to raise money next year. They take too much in fees, and we can probably do this on our own, on our own online store.

I'd like to focus on making a custom case next year and more artwork on the PCB.

After two years, I think we’re capable of doing this ourselves, but Kickstarter lends a certain amount of trust/caché that random people trust. We’ll see how it goes next time.

Thanks!


Thank you for supporting this crazy thing. It’s such a pleasure to see people using our hardware and enjoying our work. It’s been a fun ride building this with you, and we hope to start a new badge project in a few months. We just need to sleep first ;)

Special Thanks

To Corie, Charles, James, Jon, Jeri, Josh, Seth, and everyone else who came over, soldered, and put things into boxes. To our friends and significant others who had to deal with us during the build process, and to everyone for contributing and making the project happen.

To Whitney, and the entire [redacted] Slack where we make things happen behind the scenes.

To the many friends in #badgelife, including those that allowed us to group-by batteries (Cmdc0de),  CPUs (and!Xor), and SAO connectors (TwinkieTwinkie).

See you next year.

John Adams

Published 3 months ago