> In this work, we present the first text-to-image diffusion model that generates an image on mobile
devices in less than 2 seconds. To achieve this, we mainly focus on improving the slow inference speed
of the UNet and reducing the number of necessary denoising steps.
As a layman, it's impressive and surprising that there's so much room for optimization here, given the number of hands on folks in the OSS space.
> We propose a novel evolving training framework to obtain an efficient UNet that performs better
than the original Stable Diffusion v1.52 while being significantly faster. We also introduce a data
distillation pipeline to compress and accelerate the image decoder.
> As a layman, it's impressive and surprising that there's so much room for optimization here, given the number of hands on folks in the OSS space.
There's only so many folks in OSS space that are capable of doing work from this angle. There are more who could be micro-optimizing code, but the most end up developing GUIs and app prototypes and ad-hoc Python scripts that use the models.
At the same time, the whole field moves at ridiculously fast pace. There's room for optimization because the new model generations are released pretty much as fast as they're developed and trained, without stopping to tune or optimize them.
Also, there must be room for optimization given how ridiculously compute-expensive training and inference still is. Part of my intuition here is that current models do roughly similar things to what our brains do, and brains manage to do these things fast with some 20-50 watts. Sure, there are a lot of differences between NN models and biological brains, but to a first approximation, this is a good lower bound.
It isn’t obvious to me that these models produce something similar to our brains’ output. We can imagine images of course, but the level of quality is hard to define, and it is really hard and time consuming to save the output of an imagined image.
People paint or draw imagined images, but that’s a slow process and there’s a feedback loop going on throughout the whole thing (paint a bit, see how it looks, try a little happy tree, didn’t work out, turn it into a cloud). If we include the time spent painting and reconsidering, image generation using humans is pretty expensive.
An iPhone battery holds tens of watt-hours. A painting might take hours to make (I don’t paint. A couple hours is quick, right?), so if the brain is burning tens of watts in that time, the total cost could be in the same ballpark as generating images until your battery dies. But of course it is really hard to make an apples-to-apples comparison here because the human spends a lot of energy just keeping the lights on while bandwidth is limited by the rate of arm-movement.
I'm sad that Carmack decided to (as I understand it) focus away from LLMs because they're "already getting enough eyes" -- it feels like he wants to make a novel paradigm shift kind of contribution, but his magic power has always seemed to me to be a capability of grasping a huge amount of technical depth in detail, and seeing past the easy local optima to the real essence of the computation being done, and finding ways to measure and squeeze everything out of that.
Carmack could possibly get us realtime networked stable diffusion text to video and video to video at high resolution, maybe even on phones. It will probably happen anyway, but it might take 5+ extra years, and there'll probably be a ton of stupid things we never fix.
Clear and legible pricing is essential. I took a look at the event page in question [1]. The "ea." seems like something that can be overlooked, especially given the comparatively miniscule font size (8px versus 14px for the price) [2].
On the checkout page, the fact that it is per ticket is clearer, but there's no total/subtotal information. Looking at the "Terms Of Use And Privacy Policy" [4], there's also this fun line:
> You further understand and agree to pay service fees and delivery fees, which cover the costs of our operations, including the 100% buyer guarantee outlined below, the checkout security services, customer service, and the delivery of the tickets.
...and those fees are supposed to be shown during checkout. [5]
Hi there, thanks for the feedback! I overlooked testing with scrollbars set to always show, as Firefox recently started following macOS/Windows system settings rather than using its own setting. I've updated the CSS to use `100%` to rectify this. For this specific post, a bare link was also causing additional horizontal scroll, so I addressed both in https://github.com/devadvance/devadvance.github.io/commit/6a....
Thanks for posting this link. I saw this submission and remembered that Chromium had a terminal rendering mode, but couldn't manage to find this page, since I wasn't searching Ozone specifically.
The fork here goes even further, which is really cool.
> The fork here goes even further, which is really cool.
The fork here goes incomparably further. Libcaca just produces an ‘ASCII art’ approximation of an image, not capable of handling text. So, while the photo is too small to see it, the text on the terminal is gibberish. The libcaca backend was done as an backend interoperability exercise, and run on a real terminal just for fun.
Highly encourage exploring Flutter as well, at least for new platforms. Lottie animations would pair nicely, as would any future goal of expanding to web or desktop.
Thanks! Flutter does sound good, I've just started looking into it. Am I right in thinking we would rebuild the UIKit components / views in Flutter and then would be able to use the same UI across platforms (iOS, Android + Web)? And it would still be possible to include the native Apple APIs?
Highly encourage folks to take a look at Valetudo. I recently installed it on a Dreame Z10 Pro and it works well. Even without various integrations, I added a DNS record for it and can happily go to http://vacuum/ to control it locally.
For stainless, I've found it to be a combination of making sure there's enough heat before adding food, being OK with using a bit more fat (e.g., oil, butter) than I initially expect, using the right utensil while cooking, and deglazing as necessary.
I previously worked at Intersection, an out-of-home (OOH) advertising company. Intersection does digital screens (displays and kiosks) in addition to traditional formats.
This is all public information that was previously shared, so it may be out of date: the devices are Linux (Ubuntu) [1], running an advertising player (Broadsign) [2], which has Chromium Embedded (CEF) for web content [3]. The kiosk app was written in React [4]. It does things like load most data ahead of time to provide that "instant" feeling [5].
For folks wondering about the public nature of this data: SFMTA separately publishes a full data set daily: https://data.sfgov.org/Transportation/SFMTA-Parking-Citation...