The main difference is that while you can get Nexus to list all tools, by default the LLM accesses tools by semantic search — Nexus returns only the relevant tools for the what the LLM is trying to accomplish. Also, Nexus speaks MCP to the LLM, it doesn't translate like litellm_proxy seems to do (I wasn't familiar with it previously).
Yeah they definitely belong in the same space. Nexus is an LLM Gateway, but early on, the focus has been on MCP: aggregation, authentication, and a smart approach to tool selection. There is that paper, and a lot of anecdotal evidence, pointing to LLMs not coping well with a selection of tools that is too large: https://arxiv.org/html/2411.09613v1
So Nexus takes a tool search based approach to solving that, among other cool things.
Disclaimer: I don't work on Nexus directly, but I do work at Grafbase.
It’s a different scale entirely. I find copilot regularly typing around a quarter of my code for me, more in tedious cases involving any kind of patterned regularity in the code structure. It helps chew through the verbosity of so many different kinds of things. When the “boilerplate”ish aspects of the problem can fill themselves in, I spend less time in “typing out the idea” mode and can keep complicated algorithms/visualizations fresh in my head for more percent of my session: fewer forgotten glimpses of some mental visualization.
I liken it to typed holes, if you’re at all familiar with that. Copilot gives me new ways of turning skeletons of pseudocode into shippable modules, it lets my thinking be coarser for longer.
The real magic is that these models are getting faster and smarter, to the point where copilot can fill out some pattern matching code _with its bodies_ before I could have finished typing the keys that invoke the “generate match skeleton” macro that still would have had me visiting each match arm manually.
I can clarify one point, relative to migration rollbacks. We indeed chose not to implement down migrations as they are in most other migration tools. Down migrations are useful in two scenarios: in development, when you are iterating on a migration or switching branches, and when deploying, when something goes wrong.
- In development, we think we already have a better solution. Migrate will tell you when there is a discrepancy between your migrations and the actual schema of your dev database, and offer to resolve it for you.
- In production, currently, we will diagnose the problem for you. But indeed, rollbacks are manual: you use `migrate resolve` to mark the migration as rolled back or forward, but the action of rolling back is manual. So I would say it _is_ supported, not just as convenient and automated as the rest of the workflows. Down migrations are somewhat rare in real production scenarios, and we are looking into better ways to help users recover from failed migrations.
No question that there are cases where down migrations are a solution that works. In my experience though, these cases are more limited than you might think. There are a lot of presuppositions that need to hold for a down migration to "just work":
- The migration is reversible in the first place. It did not drop or irreversibly alter anything (table/column) the previous version of the application code was using.
- The up migration ran to the end, it did not fail at some step in the middle.
- The down migration actually works. Are your down migrations tested?
- You have a small enough data set that the rollback will not take hours/bring down your application by locking tables.
There are two major avenues for migration tools to be more helpful when a deployment fails:
- Give a short path to recovery that you can take without messing things up even more in a panic scenario
- Guide you towards patterns that can make deploying and recovering from bad deployment painless, i.e. forward-only thinking, expand-and-contract pattern, etc.
We're looking into how we can best help in these areas. It could very well mean we'll have down migrations (we're hearing users who want them, and we definitely want these concerns addressed).
So future features notwithstanding, is the typical Prisma workflow that if a migration failed during a production deploy the developer would have to manually work out how to fix it while the application is down?
As of now there is no strong opinionation in the tool — you could absolutely maintain a set of down migrations to recover from bad deployments next to your regular migrations, and apply the relevant one (manually, admittedly) in case of problem.
Sure we do that before every release as well. But a rollback is a much less invasive surgical fix compared to a full database restore. You're down while the new db instance spins up and any writes in the meantime will be lost.
You can also test migrations against a restored prod database snapshot but again there's no guarantee some incompatible data hasn't been inserted in the meantime.
The first thing I've looked at on your site is how migrations work. Because honestly, I think that's one of the best things about Django. They just got it right, and as you say, not many other tools get close.
I wonder if you have looked at how it works. Because they have put in something like a decade to make it work and it's very powerful and a joy to use.
Down migrations are indeed very useful and important once you get used to it. First and foremost they give you a very strong confidence in changing your schema. The last time I told someone who I helped with django to "always write the reverse migration" was yesterday.
No way you can automatically resolve the discrepancies you can get with branched development. Partially because you can use migrations to migrate the data not just to update the schema. It's pretty simple as long as we're just thinking about adding a few tables or renaming columns. You just hammer the schema into whatever the expected format is according to the migrations on that branch. But even that can go wrong: what if I introduced a NOT NULL constraint on a column in one of the branches and I want to switch over? Say my migration did set a default value to deal with it. Hammering won't help here.
The thing is that doing the way Django does it is not that hard (assuming you want to write a migration engine anyway). Maybe you've already looked at it, but just for the record:
- they don't use SQL for the migration files, but python (would be Typescript in your case). This is what they generate.
- the python files contain the schema change operations encoded as python objects (e.g. `RenameField` when a field gets renamed and thus the column has to be renamed too, etc.).
- they generate the SQL to apply from these objects
Now since the migration files themselves are built of python objects representing the needed changes, it's easy for them to have both a forward and the backward migration for each operation. Now you could say that it doesn't allow for customization, but they have two special operations. One is for running arbitrary SQL (called RunSQL (takes two params: one string for the forward and one for the backward migration) and one is for arbitrary python code (called RunPython, takes two functions as arguments: one for the forward and one for the backward migration).
One would usually use RunSQL to do the tricky things that the migration tool can't (e.g. add db constraints not supported by the ORM) and RunPython to do data migrations (when you actually need to move data around due to a schema change). And thanks to the above architecture you can actually use the ORM in the migration files to do these data migrations. Of course, you can't just import your models from your code because they will have already evolved if you replay older migrations (e.g. to set up a new db or to run tests). But because the changes are encoded as python objects, they can be replayed in the memory and the actual state valid at the time of writing the migration can be reconstructed.
And when you are creating a new migration after changing your model you are actually comparing your model to the result of this in-memory replay and not the db. Which is great for a number of reasons.
Yep we looked at Django ORM as an inspiration. I unfortunately don't have the bandwidth right now to write a lengthy thoughtful response, but quickly on a few points:
- The replaying of the migrations history is exactly what we do, but not in-memory, rather directly on a temporary (shadow) database. It's a tradeoff, but it lets us be a lot more accurate and be more accurate, since we know exactly what migrations will do, rather than guessing from an abstract representation outside of the db.
- I wrote a message on the why of no down migrations above. It's temporary — we want something at least as good, which may be just (optional) down migrations.
- The discrepancy resolving in our case is mainly about detecting that schemas don't match, and how, rather than actually migrating them (we recommend hard resets + seeding in a lot of cases in development), so data is not as much of an issue.
Well, of course I don't know about the internals, but having used Django migrations for a decade now (it used to be a standalone solution called "South" back then), I haven't really run into any inaccuracies and can't really imagine how those could happen. As far as I can see, the main difference is that they are storing and intermediate format (that they can map to SQL unambigously) while you immediately generate the SQL.
Django doesn't try (too hard) to validate your model against the actual DB schema. Because why would it? You either ran all the migrations and then it matches or you didn't and then you have to. (Unless you write your own migrations and screw them up. But that's rare and you can catch it with testing.) While your focus then seems to be to check if the schema (whatever is there in the db) matches the model definition. Based on my experience (as a user) this latter is not really something that I need help with.
Data is actually an issue in development and hard resets + (re)seeding is pretty inconvenient compared to what django provides. E.g. in my current project we're using a db snapshot that we've pulled from production about two years ago (after thorough anonymization, of course). We initialize new dev environments and then it gets migrated. It probably takes about half a minute to run as opposed to about 2 seconds of back migrating 2-3 steps.
It makes a lot of sense. I have a fair amount of Rails experience with ActiveRecord, and it was also my impression that the database schema drifting in development is rarely a problem, but I now think it's a bit of a fuzzy feeling and discrepancies definitely sneak in. The main sources of drift in development would be 1. switching branches, and more generally version control with collaborators, 2. iterating on/editing of migrations, 3. manual fiddling with the database
One assumption with Prisma Migrate is that since we are an abstraction of the database, and support many of them, we'll never cover 100% of the features (e.g. check constraints, triggers, stored procedures, etc.), so we have to take the database as the source of truth and let users define what we don't represent in the Prisma Schema. On SQL databases, we let you write raw SQL migrations for example, so you have full control if you need it.
The development flow in migrate (the `migrate dev`) command is quite pedantic: it will check things like migrations missing from your migrations folder but already applied to the dev database, or migrations that were modified since they were applied (via a checksum) and guide you towards resolving the problem. That can happen because of merges, but even more commonly when you are just switching branches locally, or editing migrations.
We're also looking into ways to integrate with your CI and review process to provide more insight, for example when your branch gets stale and you need to "rebase" your migrations on the migrations from the main branch. It's something we are actively exploring, and we're more than happy to get your feedback and ideas on github/slack :)
My personal view is that _some level_ of consciousness of the migration process (schema and data) will always be required from developers on large enough projects. What we can do is build tools that help people getting their migrations right, but it can't be completely automated. There's a lot more to do, now that we have a stable, production ready foundation.
I like to think about a single migration being a function and a migration history multiple functions being composed.
Category theory's main subject (I think) is composition and a lot of good ideas can be taken from there (see this paper: "Functorial Data Migration"[1]).
There is an obvious relationship between a schema and the data, and a migration operates on both. The schema might be versioned, but unless you use a Schema-less database (ie: MongoDB) and handle the data versioning application side, your data is not.
Any document/graph-based database can avoid this problem by assigning a version to your documents (like apiVersion+kind in Kubernetes), then the application should know how to process each version of the data.
In an SQL database, it would require different tables which complexify the data model.
IMHO, in every case, you need to handle some of the logic application side or maintain an history of your data (bye-bye storage space?).
Ooh it's interesting you mention David Spivak's work, I got interested into it recently. Some of these ideas are being commercialized by https://conexus.com/ (he's a co-founder). You might also be interested by Project Cambria[1] for novel ideas about data and schema migrations based on lenses.
We haven't gone in the fancy direction yet, because we wanted to first have the tool our users told us they were missing - namely a migration tool similar to what other ORMs have, but playing on the strengths of the Prisma schema. But these novel ideas about migrations are definitely something we always read with a lot of interest and love to discuss — I hope we get to implement something radical like that some day.
I am really enjoying lean 3 as a theorem prover; lean 4 as a programming language is a really intriguing/enticing prospect. The "functional but in place" paradigm is something quite new, as far as I can tell. For reference, see the "Counting Immutable Beans" paper[1] by the designers of the language, that details how the runtime makes use of reference counts to do destructive updates in pure code whenever possible.
Lean has an array type, which the docs say is implemented much like a C++ vector or Rust Vec. But data types in functional programming languages all expose an immutable interface, so what happens when someone changes the list in two different ways? Is a new copy of the array made, is some variant of a persistent array used, or does the program just fail to compile?
Thanks. I assumed that option would be a major footgun (since accidental copies can be very expensive) but if it works for Swift, it can't be that bad.
A similar-but-different approach exists in (largely Dutch) research around uniqueness typing, which uses static analysis guarantees to allow for destructive updates wherever the compiler can prove something is uniquely referenced. To my knowledge, efforts have been made to integrate that approach into GHC.
I think the novel thing is that this is doing it dynamically, which bypasses the static analysis and makes it applicable to situations where static analysis wouldn't be useful.
As I understand it, it does some static analysis still. The big idea with the Perseus algorithm, I think, is that instead of lending a reference at a function call, it is given. This leads to numerous optimizations where memory can statically be reused. They still do dynamic checks that an object has only one reference, too, when it's not statically known.
Are you referring to Linear Haskell[0] or some other effort? Linear Haskell can definitely express destructive updates, but IIRC it needs unsightly stuff like constructors which take continuations and method signatures along the lines of
length : Array a -o (Array a, Unrestricted Int)
which have to thread the unique value through them, since there's no notion of borrowing.
I'm talking about uniqueness typing. It's similar to linear types, but not the same. If you search the term, you can find a few research papers from Dutch universities, particularly around Clean, a programming languages which uses uniqueness typing instead of monads for IO.
I can't claim to know the details, but my understanding is that the interaction of linear typing with dependent types is an open research problem (see the quantitative type theory papers, for example). It's also a burden on the user rather than the runtime, so it's a tradeoff.
Is it still a big burden if the theorem prover can automate checking that a function can be evaluated without ref counting / garbage collector? I understand that it's a very hard problem though.
I'm trying to understand why dependent types can't represent linearity since you can model Nat at type level in dependent types. Dependent types can do capability modeling at the type level and linearity (or affinity) seems like a capability. From this thread (https://www.reddit.com/r/haskell/comments/20k4ei/do_dependen...) it seems like it can be done. I'm looking for something more formal to explain the orthogonality (or simulatability) of linear types inside a dependent type system.
From my understanding, dependent types can model linearity, but you'd need to use that model type system rather than the 'native' type system (similar to how, for example, Bash can model OOP (e.g. using a mixture of naming conventions, associative arrays, eval, etc.), but isn't natively OOP). If we go down that route, we're essentially building our own programming language, which is inherently incompatible with the dependently-typed language we've used to create it (in particular: functions in the underlying language cannot be used in our linear-types language, since they have no linearity constraints).
A common example is a file handle: I can prove that the 'close' function will produce a closed handle, e.g.
close : Handle Open -> Handle Closed
I can prove that those handles point to the same file:
close : (f: Filename) -> Handle f Open -> Handle f Closed
I can prove arbitrary relationships between the input values and the output value. Yet nothing I do restrict the structure of a term, like how many times a variable is used, to prevent e.g.
foo h = snd (close h, readLine h)
For that sort of control, we need to build the ability to track things into the term language itself; either by adding linear types (or equivalent) to the language, or building a new incompatible language on top (as I mentioned above).
Cool. Actually I should know more about this since I took a lecture series with Krishnaswami on this topic. He combines the two type systems in a way that makes sense and can produce a programming language.
But yes, for the linearity control you'd need to model variables within the type system and indirect to a function handling h's resource relationship instead of using it directly.
Formally I would like to compare linear types and dependent types and their ability to encode or not each other. I don't know what the tools for that are.
I checked recently, and the Rust implementation of arrow — parquet as well — is sadly still not usable on projects using a stable version of the compiler because it relies on specialization.
There are some Jira issues on this, but there doesn't seem to be a consensus on the way forward. Does someone have more information, is the general idea to wait for specialization to stabilise, or is there a plan, or even an intention, to stop relying on it?
Last I checked when I tried to the library last the blocker on stable was packed_simd, which provides vector instructions in Rust. I can imagine the arrow/datafusion-guy isn't too keen on dropping vectorized instructions as that would be letting up a huge advantage and I'm imagine it's used liberally throughout the code.
As for stablizing packed_simd, It's completely unclear to me when that will land in stable rust. I recently had a project where I just ended up calling out to C code to handle vectorization.
> I recently had a project where I just ended up calling out to C code to handle vectorization.
packed_simd provides a convenient platform independent API to some subset of common SIMD operations. Rust's standard library does have pretty much everything up through AVX2 on x86 stabilized though: https://doc.rust-lang.org/core/arch/index.html --- So if you need vectorization on x86, Rust should hopefully have you covered.
If you need other platforms or AVX-512 though, then yeah, using either unstable Rust, C or Assembly is required.
The main blocker is specialization as it's being used in several places including parquet and arrow crate. As this feature is unlikely to be stabilized, we'll need to replace it with something else but so far it is challenging.
Thanks for clarifying that the plan is to stop relying on it :) Is there a specific place/issue/ticket to discuss this? If time allows, I would be interested in helping out.
