What Differentiates Us: Slight Tradeoffs

Authors
  • Name
    Colman Humphrey
    Twitter
Published on

Of course we could be boring and fill this post with stuff like "we're good and fast and our competitors are bad and slow". It's more interesting to focus on what tradeoffs we're making. Naturally we think these are good tradeoffs — otherwise we wouldn't have picked them — so take all this with a grain of salt.

In any given company, every tool will have sweet spots, and areas where it's worth using but is not perfect. This post clarifies where Slight fits in.

Our Position within the Stack: The Interaction Between Data Analysts and Their Stakeholders

This is a big one, and influences all the tradeoffs in this post.

We're acutely focused on the interaction between domain experts and the data team. Say your data team has written a query or script to solve a request: often you have to do a lot of extra work to actually deliver the result to the requesting team.

We want to empower your data analysts to deliver their work, and for all the teams in your business to get as much value as they can from this work. I'll run though a few examples.

Our finance team wants to pull the results of a complex query into a spreadsheet. They want to specify the date range, and they may want to cut it down further, e.g. to a subset of clients. Writing such a query is in the wheelhouse of our data analysts, but delivering this work can be annoying. Instead, the data analysts can write the query in Slight, add some parameters and as much or as little documentation as makes sense, and create an app. The finance team can run this app on demand, move the data into a spreadsheet with a single click, and even update the spreadsheet from there.

Our data analysts are skilled with Python or R. Customer service wants to control a new feature flag for customers, or maybe client services want the results of a regression predicting which customers are most likely to churn. Data analysts might enjoy writing the scripts for these, but it can be frustrating to find a good way to allow teams to run them. A secure and stable deployment might not be what they've trained for, or just not what they enjoy spending their time doing.

We also might want to share the results through an API: our frontend team wants to show off the company's latest (cleaned) data, a backend team wants to prototype a really quick feature that relies on a query, or an external client wants to receive their data on demand. In each case, the data analysts can be perfectly suited to solving the business logic, but deploying an API is either a massive pain, or just out of their comfort zone.

There are tradeoffs in both directions within the data stack.

In tools more focused on the domain experts, those domain experts might not need to go anywhere else or use another product. Such tools often allow complex interactions — we'll talk about this more below. They might have 80% of the functionality of spreadsheets combined with complex and beautiful graphs. For true no-code analytics where anyone can create a dashboard, Slight might not be the ideal tool. The downside of these tools is vendor lock-in, and data teams typically have to do more work here, learn a new proprietary language, deal with messy ways of defining variables, or put up with a tool that simply adds friction to their workflow.

In tools more focused on data teams, some of the hard parts of data work can be abstracted away. Notebook cells can make it easy to see how your analyses progress, no need to chuck code constantly into a REPL, or unwrap and re-wrap your SQL CTEs to see if they're right. They can help make machine learning models, or allow lots of real-time collaboration. The downside here is that these tools often produce artifacts that people outside of the data team don't enjoy interacting with, or in some cases can't without a high baseline of technical knowledge.

There are many more tools with a heavy focus on one end or the other than there are focused on the intersection, so it's certainly not obvious that this is the right choice. This is just where we've felt the pain working as data analysts and scientists.

We Don’t Need to Be the Data’s Final Stop

We're not against it, we're just not determined to be the final node in processes that involve data.

Does a given team work best in a spreadsheet? Then that's where we want them to get their data. We often see people complaining that they did a bunch of work for an internal team "only for them to then open up Google Sheets and work there!". Our opinion is that this is in fact good!

At the time of writing, we're working on having graphs in Slight. But we don't need to be the belle of the ball: if there's another tool out there producing graphs exactly the way you and your team like, then great!

A big goal of our API is to make the SQL-to-code workflow easy. While you'll be able to run code directly in Slight, there's nothing like your local IDE for exploration, and we'll have open source packages and SDKs to make this as smooth as possible.

Of course, even with all that said: we are more than happy when our UI solves your use-case.

The tradeoff is that it's nice and convenient to stay within as few tools as possible. It can also make us potentially more vulnerable to being replaced. Finally, it can complicate the pricing model.

An API Focus, or the Joys of a Simple Interface

Unlike notebooks and fully-fledged internal tool builders, our apps will have a "flat" interface. That is, you just set some values and run it — you don't run the first part and then put the result in the second part, and then put both into the fifth part etc.

A big benefit of this is that every app and graph generates a straightforward API. This allows for the API use-cases we've discussed above, and is why it's easier for us to not be your data's final stop: a simple interface allows for easier integrations. This is helpful even if your team doesn't directly need an API generated.

The simple interface also makes it easy to link to an app: you can send someone to a query with some or all of the inputs filled in, or even link them directly to a result. A flexible app with many parameters can be overwhelming at first, but between sensible defaults and provided values, users can start just by modifying the parts they need to. It also means you can compose and extend apps quickly, and use one app as the template to start another.

The tradeoff here is that the sort of interactivity we aren't great at can be extremely useful. Even if the extra interactivity of notebooks and so on can make it unclear how an API should call them, you may have less need of an API if the app already solves what you need.

Documentation

Finally, documentation is a huge focus of ours. It's central in each app you make in Slight and in each dataset you catalog in Slight. It lives among them on the same page, and you can document each parameter and column too if you want.

Just to put one issue out of mind early on: we don't force you to document! All we require is a title and a quick description — the rest (long documentation, description and documentation of variables) is all optional. There are two aspects to what we envision. First, you add all the most important things up front, anything you absolutely need people to know when running your app. Second: we make it easy for your users to request extra documentation, to "pull" it instead of you pushing it. They can ask about a specific column in a dataset or a variable in an app if they want, so anything that's confusing can be addressed as needed. We think this gives a good balance of effort to reward.

We hate far-away documentation. It's hard to keep bouncing back and forth, it can't be tied easily to specific parts of an app or dataset, it's hard to keep it up to date, the links to it always go missing, somehow multiple versions often exist, it's not easy to request updates, and access often has to be granted with a totally different process. Not too fun.

There are tradeoffs here though. The documentation can take away from a simpler interface: depending on the app, dataset or catalog, this can make the interface quite busy. And out of date or incorrect documentation can be worse than no documentation.

Summary

It's not that there is a whole heap of tasks for which Slight is the only possible tool for the job. Instead, we think our tradeoffs strike the right balance for a substantial set of common workflows that involve both data teams and domain experts.

We won't pretend we're the perfect solution to every data problem. But if delivering data work to the whole team is an area where people are doing a lot of manual work, or your current tools are painful, we'd be happy to hear from you. Email us at team@slight.co or fill out our quick form, whichever you prefer.

Related Articles