Both make it pretty dead simple to deploy. AWS Copilot being the "more powerful" of the two, but still dead simple to use compared to CDK, Cloud Formation, or writing Terraform or Pulumi scripts.
I think usually this is difficult unless you "choose your own adventure". My attempt to do this is through various side projects, one of which is going relatively well (I at least shipped it!): https://techposts.eu
It's a tech news and jobs website focusing on Europe. I see it as an attempt to showcase and highlight all the good stuff going on in Europe that otherwise flies under the radar.
We need more optimism here, that's the idea anyway!
Love this project - just registered. I will bookmark and try to be active - we need this in Europe: we have both startups, companies, innovation and awesome people, but we don’t have such an outlet to share these things. Keep up the good work!
Thanks for the kind words! If you have any feedback please let me know. My current plan is to focus expanding the jobs functionality some more to allow more filtering, because I'm posting more jobs than I was when the project started... and then possibly look at tracking companies and funding rounds too. But at the same time I don't want to neglect the "news" part of the site.
The hard part I think it’s attracting users. The jobs section is nice to have, but the users come mostly for the news section, I guess?! i know that’s why I am there. Investing in filtering it’s worth when you have a big list of sort to filter.
I stumbled upon some great reddit posts this year with reading suggestions, and compiled my own "humanity is fucked" themed reading list, which included:
* Mercy of Gods by James S.A. Corey
* The Light Pirate by Lily Brooks-Dalton
* Oryx and Crake by Margaret Atwood
* Dawn by Octavia Butler
I then diverged from this list (I have more) to re-read (though it's not such a great divergence):
* If This Is a Man / The Truce by Primo Levi
Other books I enjoyed reading this year in no particular order:
* Tau Zero by Poul Anderson
* Machine Vendetta by Alastair Reynolds
* Elysium Fire by Alastair Reynolds
* Aurora Rising by Alastair Reynolds
* Shadow of the Silk Road by Colin Thubron (loved this)
* The Lord of the Rings (the god knows how many times re-read)
* The Centauri Device by M. John Harrison
* Future's Edge by Gareth Powell
* Blueshift by Joshua Dalzelle
* The Heart of a Continent by Francis Younghusband (I didn't quite manage to finish it, but it was a fascinating read nonetheless)
A decoder predicts the next word (token) to iteratively generate a whole sentence. An encoder masks a word in the middle of a sentence and tries to predict that middle.
The original transformer paper from google was encoder-decoder, but then encoder BERT was hot and then decoder GPT was hot; now encoder-decoder is hot again!
Decoders are good at generative tasks - chatbots etc.
Encoders are good at summaration.
Encoder decoders are better at summaration. It’s steps towards “understanding” (quotes needed).
It's an alternate architecture of LLMs, they actually predate modern LLMs. An encoder-decoder model was actually the model used in the "Attention if all you need" paper that introduced the transformer and essentially gave birth to modern LLMs.
A encoder-decoder model splits input and output. This makes sense for translation tasks, summarization, etc. They're good when there's a clear separation of "understand the task" and "complete the task", but you can use it for anything really. A example would be send "Translate to english: Le chat est noir." to the encoder, the encoder processes everything in a single step, that is understand the task as a whole, then the output of the encoder is fed to the decoder and then the decoder runs one token at a time.
GPT ditches the encoder altogether and just runs the decoder with some slight changes, this makes it more parameter efficient but tends to hallucinate more due to past tokens containing information that might be wrong. You can see it as the encoder running on each token as they are read/generated.
Edit: On re-read I noticed it might not be clear what I mean by past tokens containing wrong information. I mean that for each token the model generates a hidden state, those states don't change, so for example an input of 100 tokens will have 100 hidden states, the states are generated at once on the encoder model, and one token at a time on the decoder models. Since the decoder doesn't have the full information yet, the hidden state will contain extra information that might not having anything to do with the task, or even confuse it.
For example if you give the model the task "Please translate this to chinese: Thanks for the cat, he's cute. I'm trying to send it to my friend in hong kong.". For a enc-dec model it would read the whole thing at once and understand that you mean cantonese. But a decoder only model would "read" it one token a time it could trip in several places, 1. assume chinese means mandarin chinese not cantonese, 2. assume that the text after "cute." it's something to also translate and not a clarification. This would have several token worth of extra information that would confuse the model. Models are trained with this in mind so they're used to tokens having lots of different meanings embeded in them, then having later tokens narrow down the meanings, but it might cause models to ignore certain tokens, or hallucinate.
Hi, I'm not on the t5 Gemma team but work on gemma in general.
Encoder Decoder comes from the original transformers implementation way back in 2017. If you look at figure 1 you'll see what the first transformer ever looked like.
Since that time different implementations of transformers use either just the encoder portion, or the decoder portion, or both. Its a deep topic so hard to summarize here, but Gemini explains it really well! Hope this gets you started on some prompting to learn more
The announcement of the original T5Gemma goes in some more detail [1]. I'd describe it as two LLMs stacked on top of each other: the first understands the input, the second generates the output. "Encoder-decoder models often excel at summarization, translation, QA, and more due to their high inference efficiency, design flexibility, and richer encoder representation for understanding input"
You can use everything as a radiator, but you can't use everything as a radiator sufficiently efficient to cool hot chips to safe operating temperature, particularly not if that thing is a thin panel intentionally oriented to capture the sun's rays to convert them to energy. Sure, you can absolutely build a radiator in the shade of the panels (it's the most logical place), but it's going to involve extra mass.
You also want to orient those radiators at 90 degrees to the power panels, so that they don't send 50% of their radiation right back to the power panels.
reply