CGI is a terrible company known for going over budget and under delivering on a ton of projects. Not that many companies bidding on large government contracts unfortunately
I worked there for 3 years as a graduate and I can absolutely agree with this comment, the guilt of knowing my project (which was a failure from start to finish) was funded by taxpayer money drove me away from the company and made me swear to never work at another big consultancy again.
The post tells you there is a single point of failure: if you wanted to DDOS OpenAI, you'd target write-heavy operations.
For that reason, I find it actually bold that they disclosed it, and I appreciate it.
The article reminded me of a similar post about MySQL use for Facebook from the Meta team, which had the same message: big database servers are powerful workhorses that scale and are very cost-effective (and simpler to manage than distributed setups where writes need be to carefully orchestrated - a very hard task).
The two core mesages of both articles combined could be read as: 1. big DB servers are your friend and 2. keep it simple, unless you can't avoid the extra complexity any more.
What Facebook post are you referring to? Generally speaking, Facebook's MySQL infra has been heavily sharded for a very long time, and doesn't rely on abnormally-beefy servers. It's basically the complete opposite approach of what OpenAI is describing here.
when I joined twitter in 2011 there was a single mysql master user (not tweets) database and a few dozen read replicas. it was writing about 7000 updates per second and during bursts it would go too high for the single-threaded replication in mysql at the time to keep up with the master which would cause replication lag and all kinds of annoying things in the app. you just have to pick the right time to make the switch before it is an emergency.
Postgres setups are typically based on physical replication, which is not an option on MySQL. My testing shows the limit to be about 177k tps with each transaction consisting of 3 updates and 1 insert.
Tried to play dota 2 on Linux, it has a bad memory leak that made the whole system hang after an hour or two. Plus it seems to get worse fps on Vulcan, but I read that it might just be a bad implementation in my card (6650 xt)
I often find that Linux native games (Dota is one such) run worse than Windows games running under Proton. For many of them, you can just force it to play via Proton and go along with your day, but the Valve competitive ones don’t work that way since VAC will cry about it.
Also tried that, but I did not noticed the memory leak problem, since I only played about 1 hour.
There are other minor issues about using Linux, for example, my input method does not work in the Steam chat window, but works everywhere else. Anyway, Linux is much more cleaner than Windows and it is a overall better development environment, I prefer it over Windows now.
Using WebKit (even JavaScriptCore) and working with Apple wouldn't be the worst thing to my eyes as a Firefox/FOSS user. Duopoly isn't great but it's better than a monopoly.
To them, it’s bad, to us, it’s good. Good and bad are subjective when one side refuses science. You can show them data until you’re blue in the face but they won’t understand it, they won’t listen to reason because they are on a mission to remake the US like their racist grandfathers liked it. All workers report to the floor.
I've been disappointed by Nats. Core Nats is good and works well, but if you need stronger delivery guarantees you need to use Jetstream which has a lot of quirks, for instance it does not integrate well with the permission system in Core Nats. Their client SDKs are very buggy and unreliable. I've used the Python, Rust and Go ones, only the Go one worked as expected. I would recommend using rabbitmq, Kafka or redpanda instead of Nats.
I've had the same experience and I fixed part of the problem by writing my own Rust client, Watermelon. It's still missing a lot of features but at least I'm not blocked by weird decisions taken by upstream.
Client SDKs are often a major challenge in systems like these. In my experience, building SDKs on top of asynchronous protocols is particularly tricky. It's generally much easier to make the server-side part reliable. The complexity arises because SDKs must account for a wide range of usage patterns - and you are not controlling the usage.
Asynchronous protocols frequently result in callback-based or generator-style APIs on the client side, which are hard to implement safely and intuitively. For example, consider building a real-time SDK for something like NATS. Once a message arrives, you need to invoke a user-defined callback to handle it. At that point, you're faced with a design decision: either call the callback synchronously (which risks blocking the socket reading loop), or do it asynchronously (which raises issues like backpressure handling).
Also, SDKs are often developed by different people, each with their own design philosophy and coding style, leading to inconsistency and subtle bugs.
So this isn't only about NATS. Just last week, we ran into two critical bugs in two separate Kafka SDKs at work.
Are those recommendations based on using them all in the same context? Curious why you chose Kafka (or Redpanda which is effectively the same) over NATS.
There seems to be a few typos in the article, unless I don't understand what's going on.
In step 1, the register uses the source path 'github-events' but the payload is using 'github-pushes'.
In step 3 it says filter_query passes since payload->>'ref' is 'refs/heads/main' but we never see that filter being inserted in the service, the only filter we see is type IN (\'PushEvent\', \'PullRequestEvent\').
Was the article edited midway through and missed some pieces?
reply