Hacker News new | past | comments | ask | show | jobs | submit login

The lack of NSFW knowledge/capability makes them less useful for content moderation. I've tried to use multimodal models for categorizing images from large, mixed data sets. 95% of the input is safe for work. 4% contains nudity but is not sexually explicit. 1% contains nudity and is also sexually explicit. I'd like to categorize content so that nudity is hidden from users by default and that sexually explicit content is always hidden.

Every model I've tried so far is bad at distinguishing sexually explicit content from mere nudity, and many models are bad at distinguishing nude from non-nude. I don't know about Gemma 3 but Google's large commercial Gemini models refuse (or formerly refused; haven't tried recently) to tell me anything useful about images containing human figures. I assume that this is due to aggressive "safety" measures. On a technical basis, I assume that a model that can distinguish 10 different breeds of dog should also be able to usefully describe images of people wearing swimsuits, nude people, and people engaged in sexual intercourse.






There are models especially tuned for it even open weight ones. llms even multimodal ones are not up to the task. You know what doesn't help the discussion at all? That everyone's response is as usual just about titties.

4 months ago I tried every dedicated NSFW-image-classifier model I could find on HuggingFace or GitHub. They have a high false positive rate on certain kinds of benign content, like close up photographs of hands with painted fingernails, and a high false negative rate on artistic nude photographs. I even tried combining multiple models with gradient boosting but the accuracy barely improved; maybe everyone is training with very similar data sets. At this point I should train my own model but I was hoping to find something capable off-the-shelf, since content moderation is such a common task.

You can just finetune an open model instead of starting from scratch... that's the point of them.



Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: