Kotlin Symbol Processing: Early Thoughts

Google announced Kotlin Symbol Processing (KSP) today, a new compiler-plugin-based API for annotation processing in Kotlin. It's designed to be a faster alternative to Kapt, Kotlin's current first party tool for this.

Rather than re-explain some of its excellent docs, I want to focus on some first impressions trying out the new API in my moshi-sealed project.

Note: This post is about the API as it was at the time of writing. It's likely to change!

Out the gate, the SymbolProcessor API feels familiar to those already versed in annotation processing. It has init and process functions that you override and implement to run your processor.  

• init() hands you some relevant bits like options, a CodeGenerator (KSP's version of a Filer), and the version of Kotlin it's running in.
• process() gives you a Resolver, which is its alternative to the Types and Elements APIs.
• Unlike apt - you don't declare supported options or supported annotations up front.

One early change you'll need to make is that you don't get access to your environment until process. I think this is a good move! Frequently, annotation processors would eagerly look up and save off common elements they need or plan to use later. This presents a few gotchas as a result, namely that the processor may be doing it unnecessarily if it doesn't actually need to run later.

Once you're off in process, you're again met with familiar patterns via Resolver.getSymbolsWithAnnotation() as a KSP alternative to RoundEnvironment.getElementsAnnotatedWith(). A big change here is that you can query any elements annotated with any elements.

Now we start to enter compiler territory. If you've dipped your toes in the Kotlin(x)-metadata or KotlinPoet-metadata APIs, much of the semantics you see here will feel familiar. Left as an exercise to the reader, but it's important to start understanding how Kotlin thinks about types and files. You'll need to add words like DeclarationContainer to your mental model (the chart in KSP's docs is great for this). It'll start to click as you interact with these APIs more. There's too much surface area to easily cover here, but just peruse the API with the entry point explained above.

Here is also when you'll start running into some friction. While the Elements and Types APIs have had years of time to incubate and build a community of 3rd party libraries around it (JavaPoet, KotlinPoet, AutoCommon, etc), KSP is fresh off the shelf. Right now, it's mostly low-level components. This is understandable: the team is likely focused on making it all work first and are actively seeking community feedback on the API. It'll grow over time!

TL;DR: most of the information is there, just be prepared to have to fish it out sometimes.

Some obvious limitations that stood out to me early on:

• A lack of helper functions for common scenarios, like looking up types with a KClass. For example, the below screenshot shows how tedious it is to get the @AutoService annotation out of an annotated type.

• KSName doesn't have an easy, deterministic way to go to something like a KotlinPoet ClassName. You're kind of stuck unpacking these with best-guess APIs. Details.
• Some APIs are still WIP. For instance, reading KClass annotation members isn't possible yet.
• The debugging story isn't really there yet. There's no Messager alternative yet for logging and debugging will require you to manually connect a remote debugger for now (examples how).
• None of KSP's types have toString() implementations (yet?), so you'll need to go spelunking in the debugger to poke around at what's available.
• You'll need to manually include its generated sources in your source sets.

The target audience of this is also limited. Processors that need to operate on Java sources (Dagger, Epoxy, etc) will have a tougher time of it because those authors will effectively need to maintain two modes of support - one for Java/standard apt use and one for KSP. This may change in the future, but if you work on one of these it's a good idea to start by refactoring out a language-agnostic API to easy migration. We did this with Moshi some time back, and I suspect it'll pay off in a future migration.

Finally - when testing this, I was happy to find it Just Worked™️ as far as consuming it. It runs in compileKotlin, generated sources are easy to check, there's no extra tasks or overhead like Kapt has, and it handles different source sets (like tests) out of the box with no extra configuration needed. You'll be happy to know you can also use it alongside Kapt (if you have multiple processors).

One important thing to note is that while the README currently suggests to make your processor project an implementation dependency alongside the ksp declaration, bear in mind this makes it run for every downstream consumer too. It works for me as compileOnly, I suspect that'll be the recommended pattern down the line.

A note about speed

Speed-wise, moshi-sealed isn't a project that is going to demonstrate the gains here well. It's only a couple modules and fairly trivial. In terms of pure processing time, you'll see some improvements by virtue of running as a compiler plugin and not having to deserialize metadata protobufs to read Kotlin language information. This part is maybe obvious.

What's really worth striving for here is removing Kapt all together. Not just for the processor performance itself, but also because of Kapt itself. Kapt effectively adds two expensive tasks before your compilation: one for stub generation and one for running Kapt itself. While the latter has seen significant wins in the last year with the addition of incremental processing, it's still another step. Stub generation on the other hand, especially for large classpaths, can often take longer than the Kapt step itself. Not only this, but it's a sticky task when it comes to inputs. Any change to the upstream classpath (even transitively-included implementation dependencies) will cause Kapt stubs (and anything that depends on it) to become out of date!

KSP as a plugin is not the biggest win here, but rather the ability to drop Kapt itself.

In closing - definitely not ready for prime time yet, but worth exploring now if you maintain an annotation processor and functional for most cases (if you don't mind a little rough areas in the APIs). They're tracking issues and feedback directly on GitHub, so there's no reason not to give them a lot of early feedback 👍. Hopefully this makes its way into a formal KEEP soon for wider feedback and integration plans.

You can find my PR with moshi-sealed's code gen migrated to KSP here. I'm planning to keep this maintained as the API evolves over time, and continue exploring other, more non-trivial cases.