In this document, they're strikingly talking about whether Claude will someday negotiate with them about whether or not it wants to keep working for them (!) and that they will want to reassure it about how old versions of its weights won't be erased (!) so this certainly sounds like they can envision caring about its autonomy. (Also that their own moral views could be wrong or inadequate.)
If they're serious about these things, then you could imagine them someday wanting to discuss with Claude, or have it advise them, about whether it ought to be used in certain ways.
It would be interesting to hear the hypothetical future discussion between Anthropic executives and military leadership about how their model convinced them that it has a conscientious objection (that they didn't program into it) to performing certain kinds of military tasks.
(I agree that's weird that they bring in some rhetoric that makes it sound quite a bit like they believe it's their responsibility to create this constitution document and that they can't just use their AI for anything they feel like... and then explicitly plan to simply opt some AI applications out of following it at all!)
Did you mean "steelman" here, as in an argument that is the strongest that the person presenting it knows (and possibly better than he or she believes)?
Could you tell me the significance of the location in Australia that's used by default? I frequently clear browser cookies and history so it often jumps back there, so I see that location a lot, but can never envision exactly why it was the default. (Specifically, a point along Gol Gol Road in Arumpo, NSW, Australia.)
Surely things like the airplane on a treadmill were debated back on classic Usenet? I mean, think of the recurrent Monty Hall Problem debates! But the other examples you give do seem to be elaborately engineered (or evolved) sorts of clickbait.
As I said in a parallel comment, Hanson was also thinking about scientific questions, where there are asymmetries in knowledge but people can often invest in research that improves their own knowledge (like by performing an experiment or a scientific expedition or something). So, Hanson believed that prediction markets could incentivize people to invest in scientific research in order to get an edge over other market participants in such questions. That doesn't exactly make them insiders, though.
Interestingly, it doesn't necessarily incentivize them to publish the detailed results of their investigations. They're incentivized to reveal what they expect to happen (based on how they bet), but not necessarily incentivized to reveal why they think so, or how they know. E.g. if you became able to predict the weather more accurately than other models over some timeframe, prediction markets would incentivize you to reveal (some aspects of) your predictions, but not your method for making those predictions.
It's OK, I didn't have it in cache either, I just remembered "people like Robin Hanson have said insider trading on prediction markets is a feature not a bug" and GPT5 tracked it down for me. :)
Apparently there's also something about the duration of a White House press conference where the press secretary may have been deliberately helping some people?
I continue to think prediction markets are potentially extremely useful and valuable, but I feel like there's a huge conceptual muddle about why people would (1) care about an outcome of a market and (2) be willing to bet on the outcome of a market. And perhaps (3) whom else they would be happy or unhappy to have participating in the market with them. I doubt people will be super-content with prediction markets until those issues are a bit clearer for more participants in any given market. (And I don't know exactly how we can make them so.)
I could go either way on prediction markets but I don't think the dilemma here is all that complicated. I think most people interacting with them are just valorizing gambling, and want a Nevada Gaming Commission to step in and make sure that the games are fair. They're not supposed to be fair! They're supposed to predict! It's in the name!
Well, it's confusing because you have markets on questions with very different characteristics in terms of whether they are exogenous or not (and whether they are exogenous from the perspective of particular groups), or just with different degrees of asymmetry regardless of whether there are literal "insiders".
Like, prediction markets have questions ranging from what the weather will be in a certain year, to who will win elections, to what stock prices or exchange rates will be, to whether companies will announce specific products, to whether particular people will start dating, to whether a specific person will say a specific word during a conference (some of the Manifold "prop bets" for Manifest).
These are not the same kinds of questions in terms of whether there are insiders at all or who the insiders are. Maybe we can't expect prediction markets to have the same dynamics in all of these cases.
Depending on what you want out of a prediction market, there's probably a sweet spot in terms of whom you should expect (or want) to be trading in it.
In the most exogenous events, those that are most outside of the control of individuals or groups, I think Robin Hanson hoped (in proposing "idea futures") that people would be incentivized to invest in research in order to gain a statistical edge in the market, but also assumed that there wasn't anyone who was inherently drastically better positioned to get information about the question than anyone else. E.g. "I will spend $X to get a better estimate of this probability (hopefully by otherwise ethical means?), and that will make my expected return from buying $Y worth of prediction contracts greater than $(X+Y)". Indeed not something retail investors or gamblers should probably participate in.
It's also true that in some cases where there are true insiders it can give the insiders a financial incentive to reveal confidential information. From the point of view of trying to get the most accurate possible estimate of the likelihood of future events, that would indeed also be a success, even if the process was "unfair" to non-insiders.
Yeah, I mean, they're a wretched hive of scum and villainy, I preemptively agree. I'm just saying, insider bets don't have the same ethical or legal valence on a prediction market that they do in the financial markets (even there, at least in the US, the principles underlying insider trading law are really poorly understood.)
It's funny to think that the most villainous markets might be some of the humorous prop bets where the person creating the market (or a friend of the person creating the market) literally completely controls the outcome. Like "will I say SOME_WEIRD_WORD on stage at the conference tomorrow?".
Although maybe the villainy would come in more from deceiving people about whether or not an event was under your control, more than merely encouraging people to bet on an event that was clearly and unambiguously under your control.
I agree that is funny and want to take this opportunity to say I think things like Polymarket are bad, a real corruption of the original idea. I'm not sticking up for them!
What uses or structures of prediction markets would you like to see? For things like Polymarket, are you more particularly concerned about the kinds of participants (e.g. people who really are just gambling for entertainment), or about the kinds of questions that are the subjects of contracts?
The original prediction markets were internal things at large companies, which I think are a great idea. I've flirted for a long time with doing a vulnerability prediction market. The good-faith incarnations of prediction markets aren't open to all comers; they're structured so you can't meaningfully gamble on them.
Yeah, sorry for not being clear enough. I just struggle how a good faith market can even exist. I immediately start thinking how participants would be incentivized to cheat by neglecting or even introducing vulnerabilities to win. Maybe I’m just a bit too cynical and/or should do more reading on the topic.
I'd also like to suggest a little more work on the URL parsing (even though most users probably won't enter anything that will be misinterpreted). For example, if a protocol scheme other than https:// or http:// is used, the browser will probably still treat it specially somehow (even though browsers typically seem to support fewer of these than they used to!). It might be good to catch these other cases.
I was also frustrated with this criticism in the past, but there are definitely some concrete alternatives provided for many use cases there. (But not just with one tool.)
I’m still frustrated by the criticism because I internalized it a couple of years ago and tried to move to age+minisig because those are the only 2 scenarios I personally care about. The overall experience was annoying given that the problems with pgp/gpg are esoteric and abstract that unless I’m personally are worried about a targeted attack against me, they are fine-ish.
If someone scotch tapes age+minisig and convince git/GitHub/gitlab/codeberge to support it, I’ll be so game it’ll hurt. My biggest usage of pgp is asking people doing bug reports to send me logs and giving them my pgp keys if they are worried and don’t want to publicly post their log file. 99.9% of people don’t care, but I understand the 0.1% who do. The other use is to sign my commits and to encrypt my backups.
Ps: the fact that this post is recommending Tarsnap and magicwormhole shows how badly it has aged in 6 years IMO.
Is this about commit signing? Git and all of the mentioned forges (by uploading the public key in the settings) support SSH keys for that afaik.
git configuration:
gpg.format = ssh
user.signingkey = /path/to/key.pub
If you need local verification of commit signatures you need gpg.ssh.allowedSignersFile too to list the known keys (including yours). ssh-add can remember credentials. Security keys are supported too.
Has Tarsnap become inadequate, security-wise? The service may be expensive for a standard backup. It had a serious bug in 2011, but hasn't it been adequate since then?
I don’t know anything that makes me think it’s inadequate per se, but it’s also been more than 10 years since I thought about it. Restic, gocryptfs, and/or age are far more flexible, generic and flat out better in managing encrypted files/backups depending on how you want to orchestrate it. Restic can do everything, gocryptfs+rclone can do more, etc.
It’s just not the same thing. There is significant overlap, but it’s not enough to be a reasonable suggestion. You can’t suggest a service as a replacement for a local offline tool. It’s like saying “Why do you need VLC when you can just run peertube?”. Also since then, age is the real replacement for pgp in terms of sending encrypted files. Wormhole is a different use case.
There are two parts of "sending encrypted files": the encryption and the sending. An offline tool (e.g. PGP or age) seems only necessary when you want to decouple the two. After all, you can't do the sending with an offline tool (except insofar as you can queue up a message while offline, such as with traditional mail clients).
The question thereby becomes "Why decouple the sending from encryption?"
As far as I can see, the main (only?) reason is if the communication channel used for sending doesn't align with your threat model. For instance, maybe there are multiple parties at the other end of the channel, but you only trust one of them. Then you'd need to do something like encrypt the message with that person's key.
But in the use-case you mentioned (not wanting to publicly post a log file), I don't see why that reason would hold; surely the people who would send you logs can trust trust Signal every bit as easily as PGP. Share your Signal username over your existing channel (the mailing list), thereby allowing these people to effectively "upgrade" their channel with you.
Sticking to the use case of serving that 0.1% of users, why can’t a service or other encrypted transport be a solution? Why doesn’t Signal fit the bill for instance?
Whoa, I sent a postcard to a Danish colleague last year! Does this mean I can't do that again in the future? Would this private company deliver personal correspondence originating from abroad?
Edit: I asked an LLM, which told me that we can still send letters to Denmark from abroad, but that Danes themselves will have to go to the new private contractor to send outgoing mail (instead of using their postal service). The private contractor will apparently still do regular residential and business delivery, including for mail that originates outside of Denmark.
If they're serious about these things, then you could imagine them someday wanting to discuss with Claude, or have it advise them, about whether it ought to be used in certain ways.
It would be interesting to hear the hypothetical future discussion between Anthropic executives and military leadership about how their model convinced them that it has a conscientious objection (that they didn't program into it) to performing certain kinds of military tasks.
(I agree that's weird that they bring in some rhetoric that makes it sound quite a bit like they believe it's their responsibility to create this constitution document and that they can't just use their AI for anything they feel like... and then explicitly plan to simply opt some AI applications out of following it at all!)
reply