At Cortex, we have a lot of SDKs and Swagger generated clients that could throw exceptions at any time. This could be a fault in our logic, or it could be a fault with the 3rd party service (and of course some third party SDKs have no rhyme or reason to how their exception handling works.) — but either way, when our customer wants to fetch Git commit history for a service, they definitely do not want to be greeted with a 500 and an esoteric error message.
In the past, we've seen GitHub go down during a customer demo and 3rd party integrations throwing other unexpected exceptions. Overall, it was a very crappy experience for our customers when something unexpected was happening with an integration that we have no control over.
Thus, `IntegrationValue<T>` was born. It is a type that can return any value but encapsulates all possible errors from 3rd party integrations.
Now expected and unexpected exceptions and errors are (mostly) handled gracefully and we can surface clean error messages to our customers when we cannot retry or handle them.
It's a monad. (Just kidding, but will get back to this concept at the end.)
We do all of our backend development in Kotlin, and `IntegrationValue<T>` is first and foremost a way to workaround the fact that Kotlin doesn't support checked exceptions. But even more than exceptions, it can capture unexpected errors that don't necessarily raise exceptions.
For example, if I want to fetch a service's Github repo details, but the workspace doesn't have a GitHub integration enabled, what should it return? We could return null, but what does this mean for the caller of `GithubService.gitDetails(serviceId: Long): GitDetails?`?
It could be that:
Basically, it could be one of a plethora of errors, and we don't want to callously bundle them all together in our type system as a null. Rather we should be explicit on what exactly those error types could be.
This is exactly what we need from our 3rd party integrations, where we need to surface clean error messages back to the customer as well as have tightly scoped error handling strategies, depending on the error.
One failing of a monolithic error type is that it might encompass errors that are not possible for a specific operation, but for this reason we only limit this type to code involving 3rd party integrations. But we’ll get back to this at the end.
Almost there! So now if we need all of our integration services to return possible error types, and we don't want to throw exceptions because callers are unaware, we need a way to bake errors into our type system.
Basically, the resulting type of integration service functions is either a failure (`IntegrationError`) or a success (any type `T`). And there's a handy concept from functional programming land that captures this concept: `Either<L, R>`.
Going back to our GitHub example, now we can transform that useless null version into:
So now callers of `GithubService.gitDetails` can explicitly handle specific error cases:
And to finally answer your question, `IntegrationValue<T>` is just a synonym for `Either<IntegrationError, T>`.
This is done because all integrations should have unified error types, and allows us to chain these requests with the power of monads.
Monads are a pretty abstract concept — just google "what are monads" to see literally hundreds of articles trying to explain it simply. And now n+1 articles.
But the best explanation to conceptualize them is that they "box" values. We could have some value `T`, and the "boxed" version of `T` is `IntegrationValue<T>`.
Explicitly monads can do exactly 3 things:
These methods `map` and `flatMap` look familiar because our friend `List<T>` is secretly a monad! And many other Collections. And Kotlin's nullable type `T?`. And many more.
To drive the point home, the 3 methods for `List<T>` are:
Similarly, the Either monad (IntegrationValue) has:
How IntegrationValue as a monad works is that it eagerly fails at the first time it encounters a failure (IntegrationError).
If we try to map a failed `IntegrationValue<T>` to another `IntegrationValue<U>`, the computation to map won't even happen, and the final result will be the first error we found.
And this brings us to the final point, that Arrow (our functional programming library of choice) comes with some handy utilities so that we don't need to chain a million maps and flatMaps to get any work done. We can use Arrow’s Fx library to easily chain together IntegrationValues with some syntactical sugar:
Using that magical `!`.
The Fx library is built upon Kotlin coroutines and guarantees purity of the monads with referential transparency. Which is a fancy way of saying we can’t cheat and “escape” the monad (or boxed value).
How it works is that it (in the context of the Either.fx block) tries to unbox an IntegrationValue, and if the result was an error (a Left), then it eagerly exits with that error without doing any following computation. Otherwise, it continues sequentially through the block.
It's handy just because the alternative would be a nested mess:
And you can just imagine how arbitrarily complex that could get.
Monads are especially helpful for us here at Cortex because as our codebase grows, it becomes easier and easier for errors to propagate without our knowledge. Pure monads enable us to pin down possible sources of errors and force callers to deal with all possible types of errors if they want to unbox the value they’re looking for.
Overall, monads have been an amazing abstraction for our codebase, and have allowed us to streamline the customer experience through unexpected errors.
As referenced above, one failing of encapsulating all errors in a single type has its drawbacks – either callers are forced to deal with error cases that are not possible, or they end up throwing away valid errors thinking that those errors are not possible. To my knowledge there’s not an easy way in Kotlin to “lift” error subtypes into larger error types without codegen, but methods in Haskell exist using classy lenses and prisms.
We’ve accepted this as a possible consequence to speed up development time by avoiding significant boilerplate.
If this sounds like something you’d like to work on, we’re hiring! We’re barely scratching the surface when it comes to functional programming in Kotlin and can use all the help we can get.