When trying to understand complex C codebase I've often found it helpful to rename existing variable as emojis. This makes it much easier to track which variables are used where & to take in the pure structure of the code at one glance. An example I posted previously: https://imgur.com/F27ZNfk
Unfortunately most modern languages like Rust and JS follow the XID_Start/XID_Continue recommendation (not very well-motivated imo) which excludes all emoji characters from identifiers.
You're right that writing a preprocessor would be straightforward. But while you're actively editing the code, your dev experience will still be bad: the editor will flag emoji identifiers as syntax errors so mass renaming & autocompletion won't work properly. Last time I looked into this in VSCode I got TypeScript to stop complaining about syntax errors by patching the identifier validation with something like `if (code>127) return true` (if non-ascii, consider valid) in isUnicodeIdentifierStart/isUnicodeIdentifierPart [1]. But then you'd also need to patch the transpiler to JS, formatters like Prettier, and any other tool in your workflow that embeds their own version of TypeScript...
Has this been confirmed? The original channel also posted a comparison video [1] showing what seems to be the same cylinders tested against titanium and tungsten cubes (though it's difficult to be sure they are identical)
There's also footage from another channel [2] showing a Prince Rupert's Drop bursting at 20 tons with significant damage to both the steel plate and the press.
I'm sure going to be amazed if the LLMs of the future 10y suddenly acquire the ability to physically cut just the right bit of a random surgical piece, with a precise idea of where, when and in what orientation the surgeon dug it out, all that with shitty documentation. Humans will be cheaper for a long time still.
Haha. Have you actually ever seen a surgical robot yourself? Your claim is laughable. There is no automation whatsoever in any robot on the market currently.
Almost all specialties do various technical procedures that only them really know how to do. The extreme is psychoanalytic psychiatry, which are the only ones really doing nothing with their hands (yes, interventional psychiatry is a thing...). Now, you could argue that 'yes, but most of the times it's done by techs/nurses'. Well, no. When things go south, and in all places where there is noone else to do the stuff (of which there are many) docs are on their own.
Regarding surgery, I expect it to be one of the easiest procedures to automate, actually (still quite hard, obviously). Because surgery is the only case where there's always advanced imaging available beforehand, and the environment is relatively fixed (OR).
Why do you think medical science wrt complexity is any different than applied math, which computer science essentially is? People already can use LLMs to assist them in diagnosing health issues so why would it be hard to believe that the doctors won't be using the same kind of assistance soon too?
> Why do you think medical science wrt complexity is any different than applied math
I don't think I wrote that.
Doctors already use tech assistance. I just pointed out that while we've got efficient robots for applied math, we don't have those as agents in the physical world. People who do blue collar jobs are less replaceable. Well, believe it or not, but most doctors are actually blue collar workers.
You sort of implied that with your replies across the thread. And since AI already replaced part of the CS, I was wondering why do you think this would not be the case with doctors. I'm not sure I agree it's a blue collar profession. I can easily see diagnostics being replaced with AI models.
I never wanted to imply that. But here, people frequently assume that because that's what they're used to. Diagnosis is the tip of the iceberg. Most people here aren't sick, so diagnostics are their only focus. If they get ill, they want a diagnosis. But many people are chronically ill already, and doctors spend most of their time treating, not diagnosing. Treating people is made in good part of technical procedures and practical assessments, and you need doctors for that because robots are still far behind for that kind of stuff. People actually have a completely skewed view of what a doctor is.
> People actually have a completely skewed view of what a doctor is.
It could be but treating patients also requires continuous diagnostics, result comprehension, and final assessment so this is certainly the part where AI could play the crucial role.
I don't think anyone thinking of the AI consequences on medicine is arguing that it will replace manual labor such as procedure executions or psychological support. This is obviously not possible so when I see people talking about the "AI in medicine" I read that as mostly complementing the existing work with new technology.
I did this very recently for a 19th century book in German with occasionally some Greek. The method that produces the highest level of accuracy I've found is to use ImageMagick to extract each page as a image, then send each image file to Claude Sonnet (encoded as base64) with a simple user prompt like "Transcribe the complete text from this image verbatim with no additional commentary or explanations". The whole thing is completed in under an hour & the result is near perfect and certainly much better than from standard OCR softwares.
If you're dealing with public domain material, you can just upload to archive.org. They'll OCR the whole thing and make it available to you and everyone else. (If you got it from archive.org, check the sidebar for the existing OCR files.)
Ah, yeah, that's not uncommon. I was operating on an assumption, based on experience seeing language models make mistakes, that the two approaches would be within an acceptable range of each other for your texts, plus the idea that it's better to share the work than not.
Note if you're dealing with a work (or edition) that cannot otherwise be found on archive.org, though, then if you do upload it, you are permitted as the owner of that item to open up the OCRed version and edit it. So an alternative workflow might be better stated:
1. upload to archive.org
2. check the OCR results
3. correct a local copy by hand or use a language model to assist if the OCR error rate is too high
4. overwrite the autogenerated OCR results with the copy from step 3 in order to share with others
(For those unaware and wanting to go the collaborative route, there is also the Wikipedia-adjacent WMF project called Wikisource. It has the upside of being more open (at least in theory) than, say, a GitHub repo—since PRs are not required for others to get their changes integrated. One might find, however, it to be less open in practice, since it is inhabited by a fair few wikiassholes of the sort that folks will probably be familiar with from Wikipedia.)
Is it really necessary to split it into pages? Not so bad if you automate it I suppose, but aren't there models that will accept a large PDF directly (I know Sonnet has a 32MB limit)?
They are limited on how much they can output and there is generally an inverse relationship between the amount of tokens you send vs quality after the first 20-30 thousand tokens.
I think these models all "cheat" to some extent with their long context lengths.
The original transformer had dense attention where every token attends to every other token, and the computational cost therefore grew quadratically with increased context length. There are other attention patterns than can be used though, such as only attending to recent tokens (sliding window attention), or only having a few global tokens that attend to all the others, or even attending to random tokens, or using combinations of these (e.g. Google's "Big Bird" attention from their Elmo/Bert muppet era).
I don't know what types of attention the SOTA closed source models are using, and they may well be using different techniques, but it'd not be surprising if there was "less attention" to tokens far back in the context. It's not obvious why this would affect a task like doing page-by-page OCR on a long PDF though, since there it's only the most recent page that needs attending to.
I've experienced this problem but I haven't come across papers about it. For this context, it would be interesting to compare the accuracy of transcribing one page at a time to batches of n pages.
Necessary? No. Better? Probably. Despite larger context windows, attention and hallucinations aren’t completely a thing of the past within the expanded context windows today. Splitting to individual pages likely helps ensure that you stay well within a normal context window size that seems to avoid most of these issues. Asking an LLM to maintain attention for a single page is much more achievable than an entire book.
Also, PDF size isn’t a relevant measurement of token lengths when it comes to PDFs which can range from a collection of high quality JPEG images to thousand(s) of pages of text
I recently did some OCRing with OpenAI. I found o3-mini-hi to be imagining and changing text, whereas the older (?) o4 was more accurate. It’s a bit worrying that some of the models screw around with the text.
There’s GPT4, then GPT4o (o for Omni, as in multi modal) and then GPT o1 (chain of thought / internal reasoning) then o3 (because o2 is a stadium in London that I guess is very litigious about its trademark?), o3-mini is the latest but yes optimized to be faster and cheaper
4o is going to be better for a straight up factual question
(But eg I asked it about something Martin Short / John Mulaney said on SNL and it needed 2 prompts to get the correct answer..... the first answer wasn't making anything up it was just reasonably misinterpreting something)
It also has web search which will be more accurate if the pages it reads are good (it uses bing search, so if possible provide your own links and forcibly enable web search)
Similarly the latest Anthropic Claude Sonnet model (it's the new Sonnet 3.5 as of ~Oct) is very good.
The idea behind o3 mini is that it only knows as much as 4o mini (the names suck, we know) but it will be able to consider its initial response and edit it if it doesn't meet the original prompt's criteria
You can request Markdown output, which takes care of text styling like italics and bold. For sections and subsections, in my own case they already have numerical labels (like "3.1.4") so I didn't feel the need to add extra formatting to make them stand out. Incidentally, even if you don't specify markdown output, Claude (at least in my case) automatically uses proper Unicode superscript numbers (like ¹, ², ³) for footnotes, which I find very neat.
(Also I believe these letters are 'anonymous' not in the sense that their authorship is unknown, but in that they were published under the institutional name of the Insurance Institute where he was employed, not under his personal name.)
At least you can't trivially circumvent the redaction just by copy and paste which was the case for lots of PDFs from the Obama years, such as this one:
> One key for him is that it’s only a possible threat. Our best intelligence, including in a briefing for Congress from the Biden administration Tuesday, is that the Chinese government has not actually done the things the ban fears.
> I look to Jim Himes, who is the senior Democrat on the Intelligence Committee, the ranking member. He's in what's called the Gang of Eight. He has the most exquisite access to intelligence. Jim voted against the ban. And I thought, you know what? If this guy is not seeing anything on the national security level.
> [00:44:36] There was an off the record or confidential briefing to the House Intelligence Committee. You think in that meeting, there was nothing that was very meaningful that was disclosed about TikTok?
> [00:44:45] Nothing that I had seen. Is it owned by the Chinese government? Absolutely. But is there a national security risk? I have not seen that.
Moreover your factoid is misleading because it omits the fact that the chair voted for the ban, along with most of the members. Of the 25 members on the committee, only 4 voted against.
He voted against the March bill that narrowly focuses on the TikTok issue, but voted for the April bill which also includes foreign aid for Ukraine and Israel. They are not the same bill.
> The decision by House Republicans to include TikTok as part of a larger foreign aid package, a priority for President Joe Biden with broad congressional support for Ukraine and Israel, fast-tracked the ban after an earlier version had stalled in the Senate. A standalone bill with a shorter, six-month selling deadline passed the House in March by an overwhelming bipartisan vote as both Democrats and Republicans voiced national security concerns about the app’s owner, the Chinese technology firm ByteDance Ltd. (https://www.pbs.org/newshour/politics/possible-u-s-tiktok-ba...)
Putting shit like that in an unrelated bill can be translated as a 'horseman bill' and would have been censored so fast in my country it isn't even funny. What does your supreme court do exactly?
Intelligence is kinda unnecessary at this point. A Tiktok VP already admitted to a UK parliament select committee that they used to harshly moderate anything the Chinese government found sensitive
Q90 - John Nicolson: It may happen elsewhere, and I can tell you what your official TikTok response was to this leak. You did not deny that these were instructions. In fact, you confirmed that these were instructions, but what you said was that the company had changed its policy in May 2019. Previously, you instructed your moderators to take down videos critical of China, specifically talking about incidents in Tiananmen Square, separatism in Tibet, all straight out of the Chinese Communist Party playbook. You confirmed that is what your moderators did, but your defence was that you had changed your policy in May 2019.
Theo Bertram: It is highly regrettable that that is what it was, but it is not our policy today, nor has it been for a long time.
> Google used to watermark internal emails using non-visible Unicode. It would catch people copy-pasting things to the press.
How does that even work? I assume the unique identifiers are generated along the lines of https://zws.im but do they send a different version of the same email to each unique recipient? Or does the watermark get inserted by some email client when copying text?
Most AI-generated images I've come across (I have in mind Substacks, Medium posts and other personal blogs) are purely "decorative" & having the same (lack of) purpose as stock photography - e.g. a generic photo of people exercising in an article about fitness. In informational articles I find these pointless & also maybe somewhat in poor taste. But then I'd probably feel the same way about regular stock photography.
There may be potential for AI-generated explanatory visuals though. High-quality diagrams, graphs, map of complex conceptual relationships and so on would be exciting.
That would need to be a whole new model that’s not based on pixels but these nodes and edges. All AI generated maps or diagrams look absolutely unhinged
