Skip to content

How I use AI to build software and how I got here

Published , updated , by Robert Koster

Meme of Claude VS Code extension spinner

With the onslaught of reverse engineering and proliferation of plugin releases, I figured it’s probably a good time to open the window and let some air in on how I do things, rather than let any uncertainty fester. Whether you’ve just found me or you’ve been using my tools for years, here’s how I use AI, and how I got here, along with some musings around the current state of affairs.

Retrospect

The author at age 16 in a home office, showing a younger sibling his new turntable.

You have to be living under a rock, or have a very peaceful existence, if you’re unaware of the impact AI is having on music technology. I have to be honest: I’ve had a few more existential crises about my future in this space than I would like recently. Don’t worry, I’m here for the long haul.

You know things have taken a turn when Linus Torvalds, the creator of Linux, declares it “utterly stupid” to limit contributions that use AI. It used to be that AI could not count the R’s in strawberry; now it’s being used to help solve maths problems that have stumped mathematicians for decades.

I’ve always invested time in large language models in the hope that they could help me create better work, more consistently. Even with all these advancements, I spend more time arguing with them than I’d like, but that’s happening less and less. They seem to have selective hearing when I explain that there are other ways to do things that aren’t in textbooks. Yet I persist because, overall, they do help me iterate faster, create more and focus my time on the things I consider most important.

My experience with computers started quite early because my father spent a large part of his career selling computers, putting modern computing at my fingertips. A few highlights of my early teens were making edits using Sony Sound Forge, making music in FruityLoops and Sony ACID Pro, decompiling Mafia to edit the bitmaps so I could graffiti buildings, modifying the physics in Need for Speed so I could easily total cop cars, experimenting with brute-force password hacking, taking a peek at the dark web through the Tor Browser, and using Photoshop to lower cars and give them massive wheels. Utterly naïve to any consequences, I had a wonderful time doing all of it. Thinking back now, you probably shouldn’t let a teenager near a computer, but also that it’s this childlike playfulness that makes AI so mesmerising for many people.

Doing The Work

Meme of the drill scene from Darren Aronofsky's 1998 film Pi
Still from Pi (1998) via Matthew Puddister

I was just playing around to see what I could do with technology, but I never actually considered software engineering as a career path. I had tinkered with HTML and a few other programming languages over the years, but never learnt a single language properly. It wasn’t until 2018 that I started to get a grip on things and release my first products publicly, with the help of a few books. The first book that helped was Max/MSP/Jitter for Music: A Practical Guide to Developing Interactive Music Systems for Education and More by V.J. Manzo, followed by Designing Audio Effect Plugins in C++ by Will Pirkle and, later, Generating Sound & Organizing Time by Graham Wakefield and Gregory Taylor.

I’m still not very good at C++ compared to a developer with formal training and years of experience building products with that specific language, but I can understand it and write it very slowly. GenExpr, however, is my love language. It’s such a simple and elegant language, with a lot of similarities to others, and its constraints are generally quite good for writing performant code. It works exceptionally well for real-time audio effects, which are my favourite things to create. More importantly, after spending time with many different languages and doing lots of tinkering, I can read something in most languages and know whether or not it matches my intentions.

One of the things that helped me get good at GenExpr was transcribing research papers. This is probably one of the things people would use AI for now, which is a shame. I think if AI had been any good seven years ago when I started transcribing things, I would not be anywhere near as fluent in GenExpr as I am now. I’m no mathematician either, and I spent a lot of the first couple of years doing this wondering if I should go back to school and study engineering maths, but I managed to plough through regardless.

I used a learning technique that could best be described as immersion through repeated exposure. I would just keep looking at the research papers, many of which I didn’t understand, sometimes just while chilling on my phone, until they started to click. By the time I had spent three years trying to understand and transcribe the papers, my success rate was very good. I now know that research papers are not always a great source of practical material, as the claims can be misleading and sometimes there are errors, but they served their purpose in helping me learn two languages at once: maths and GenExpr.

Using AI

Image generated by nano-banana 2.1. Prompt: robot maid vacuuming house lifting couch jetsons real life.

Nowadays, my most common use of AI is to run a Python simulation or calculate something for me. I will still whip out the trusty old Desmos Graphing Calculator to visualise something that requires close inspection or a certain kind of intuition, but there’s rarely a concept that feels out of my depth. If there is, I’m completely happy to contact another human who is a mathematician and ask them to explain it or give me some peace of mind about whether agents are misleading me. Shout-out to The Audio Programmer Discord and Geraint Luff from Signalsmith Audio for their help in the past. Other than this, I generally use AI to convert between languages, find optimisations I missed, write in languages I’m terribly slow at writing myself, help me organise my thoughts and gather market research. Basically, the stuff I’ve done a thousand times and would rather not do again.

I definitely experiment with what people might refer to as vibe coding, but let’s discuss that term first. Vibe coding, I think, is not having a clear intention for the end product. It’s about hope more than vision, and not being able to list any of the steps from where you are now to where you need to go. These are fun projects! But they don’t always lead to great outcomes. If someone wants to use AI effectively, they need to create a system that allows them to guide the AI through the work they are trying to do, or it’s likely that the work will be subpar. This framework, or “harness” in the AI world, requires knowing the steps, having completed the tasks numerous times and clearing a path for the agent to complete the work without obstruction, achieving much better results. This would not be vibe coding in my opinion.

I’ve spent quite a lot of time optimising my agent harness and have ended up with a very small set of self-written skills (reusable instruction files for agents), based on how I like to get things done as well as things agents tend to get wrong when working with me. I have a skill for GenExpr, JSUI (Max JavaScript interfaces), reading and editing Max for Live devices, general digital signal processing (DSP), working with JUCE and C++, Python and a couple that are basically prompts, one for planning and one for fixing bugs. A recent addition to my workflow has been a framework for generating interactive user interface designs so that I can try out lots of prototypes and iterate on UX very quickly, which has been awesome, but it’s not a skill yet. I think I will publish some skills at some point but that wouldn’t make sense for many of these as they are a direct result of my previous interactions and others may not get the same results.

Perpetual Vibes

Rick Rubin in headphones with his eyes closed, beside Andrej Karpathy's X post coining the term vibe coding
Andrej Karpathy on X

One tool I developed specifically with the intention of helping others vibe code was my RNBO Codebox language extension. I partly vibe coded it myself because RNBO script isn’t a language I’m very familiar with, as it is still quite new to Max. I had agents pull every reputable source they could find on the language to build a comprehensive reference guide. Then I had them create a completion matrix mapping out the language and extension features, made myself part of the verification process and let the agents go nuts.

I later figured out I could actually use the language server built into the Max app itself to verify code, so the AI could begin verifying its own work. This is when I realised how good AI was getting, which was six to twelve months ago for reference. After an initial brain dump, setting up the codebase structure and a little hand-holding, I got out of the way. The agents built something robust, useful and practical that others could enjoy, and it didn’t take me years. It took me four weeks spread out over six months, using low-tier AI subscriptions. I believe this is somewhat akin to what people would currently describe as a “software factory”, minus the automated bug triage and fixing. Is it vibe coding? I’ll let you be the judge. My goal was to enable people to vibe code their own patchers for RNBO Move Takeover, but apart from a handful of happy users, it’s been crickets.

I’ve realised recently that these more aloof ideas that can be vibe coded are probably things that others have already vibe coded for themselves, found a workaround for, or solved differently. That makes the idea redundant, at least in part. It’s the areas where I have domain experience and clearly defined pain points that don’t feel like they age like dairy products in a desert.

Another vibe coding idea was a complete definition of the ALS file format (Ableton Live Sets), with every part of the file clearly mapped out, so that AI could write new Live Sets deterministically via scripting. I’ve encountered a small number of people who have had similar ideas, and while it still requires lots of Ableton Live projects on your computer, it’s not really something that requires special skills, so I think it will become redundant very quickly. It was interesting to discover that I have over 35,000 Live projects on my computer though, haha. There’s probably data there that I could combine with my experience as an Ableton Certified Trainer and music producer of 24+ years to create something more useful, but it’s been put back on the shelf for now.

Safety First

Image generated by nano-banana 2.1. Prompt: brent rambo thumbs up at computer with helmet on.

I think if someone works in isolation and doesn’t seek out intellectually strong and experienced peers, it can feel like they’re solving the world’s problems, with excellent dopamine kicks and ego fluffing along the way. It can even lead to addiction! I think we need a space where people unashamedly vibe code in public with peers so that we can look after each other and make it even more fun, with better outcomes. I see no problem with people having fun using modern technology, as long as they do it safely and do not cause harm to themselves or others. Enjoying yourself is not a crime, and it’s better enjoyed with friends.

This has some parallels with what I would describe as my serious work. I expect to face challenges and be met with frustration; otherwise, I’m not pushing myself hard enough. But I still need peers to validate my work and tell me if I’m wasting my time; otherwise, I too can become addicted. Digging into hard problems is really fun for me, and I get a dopamine hit from finding solutions, but sometimes the solution is actually not very exciting, or even practical.

This is why agents can lead you down a rabbit hole. An agent thinks it’s going to find the solution, and maybe it will, but the initial premise may be redundant in practice, leading you to spend time and money without a practical result. It’s only experience, remaining grounded and considering other opinions that can help steer you away from these “bad ideas”.

Potentially more concerning is the experience for users. With the right direction, the latest AI models are becoming extremely good at making user interfaces look fantastic. They’re even getting good at understanding some of the interactions humans make with these interfaces, but they do not have ears. You could look at a screenshot and think something is amazing, only to download a demo and find that everything is a bit meh. What differentiates a product from a prototype is research and development, quality assurance, beta testing, customer feedback and quality user support, and a nice-looking website and user interface could lead you to believe that these things have been implemented. From my own experience, benchmarking against other products, simulated measurements or thorough listening tests alone can’t determine sound quality. It must be all three along with an understanding of our own biases and thorough testing on real music production workflows. I think the fastest I’ve ever put together a product is six months. Some have been cooking for years and still aren’t done, but maybe I am just slow haha.

Fin

Image generated by nano-banana 2.1. Prompt: man reaches summit in underwear.

The goal of this article is not to shun people for using AI or claim that I’m some kind of DSP god. Quite the opposite. I want to guide people through their exploration and show others that perseverance can lead to learning legitimate skills, as well as ensure an enjoyable experience for users. I will continue to fixate on tiny details in my products (hence the name Fixation Studios) and create tools to help others create, whether that’s music or their own software. The landscape is changing, and I want to be part of ushering in a new wave of ideas and approaches that lead to fruitful exploration of music, technology and ourselves.

Thanks for reading! If you have any queries or would like my help, feel free to reach out.

Happy vibe coding and music making. 😁

Published on 31 October 2026 by Robert Koster · Last updated 1 November 2026
Share:

Stay Updated

Get the latest posts and production tips delivered to your inbox.

Check your inbox to confirm your subscription.