I think even when the answer is "right" in some sense, it should probably come within the context of a bunch of caveats, explanations, etc.

But maybe I'm just weird. Oftentimes when my wife or kids ask me a question, I take a deep breath and start to say something like "I know what you're asking, but there's not a simple or straightforward answer; it's important to first understand ____ or define ____..." by which time they get frustrated with me.

> I think even when the answer is "right" in some sense, it should probably come within the context of a bunch of caveats, explanations, etc.

Funnily enough, this is exactly what the LLM does with these questions. So well that people usually try to tweak their prompts so they don't have to wade through additional info, context, hedging, and caveats.

So you are saying that Google should provide responses that are more likely to frustrate its users? ;)

If you’re aiming to lose weight safely the rule of thumb is 3 lbs a week. 3000kcal per pound works out to an average deficit of about 1280 calories per day. Max.

Obese people can lose a bit more under doctor supervision. My understanding is that it’s tied partially to % of body weight lost per week and partly to what your organs can process, which does not increase with body mass.

I don’t think absolute numbers are very useful here. You need around 5–10% reduction in calorie intake to get any weight-loss effect going, and I wouldn’t reduce by more than 20% (relative to weight-maintaining intake — it’s different if you’ve been seriously overeating) if you want it to be sustainable longer-term.

So for example if your weight is stable at 2500 kcal per day, I would start by reducing the intake by 250–500 kcal, but not more. If this works well for a month or two and then you want to lose weight faster, you can still reduce your intake further. You generally have to do that anyway even just to maintain the velocity, because weight loss also tends to reduce calorie expenditure.

First and foremost, you need to monitor your calorie intake against weight. Here is a useful text about that: https://www.fourmilab.ch/hackdiet/

Your body will get more efficient at whatever exercise you do to make the calories work out. So over time you’ll either have to increase your exercise or rein in the calories a bit more to achieve a sustained result.

That is assuming that you do any exercise. But yes, and the method I linked to explains how to handle any such variation in calorie expenditure regardless of its cause.

I don't see the problem with the answer, and the question is already garbage. Plus, the LLM hedges its advice with precautions.

I get a pretty good summary when I paste the question into Google. It comes up with a ballpark but also gives precautions and info on how to estimate what caloric restriction makes sense for you within the first 3 sentences.

And all in a format someone is likely to read instead of clicking on some verbose search result that only answers the question if they read a whole article which they aren't going to do.

This seems like really lame nit picking. And I don't think it passes the "compared to what?" test.

The basic problem is it says reduce "by 1,500 - 1,800" rather than "to 1,500 - 1,800" (not that that answer is much better). Yes, it's a garbage question, but the first answer is unsafe in all circumstances. The simplest solution here is to show nothing.

The question is garbage. But people will ask it with their best intentions and not know it’s garbage.

If your calorie intake is just 1500 today, it is bad advice. If your calorie surplus is 1800, it is good advice.

But I wonder, were those few words the full response? Information hiding to prove a point is too easy.

A calorie surplus of 1800/day is ~190 lbs/yr. Is that something people actually do?

Yes. You can't just say that someone eating 1800 over the recommended 2000 will perpetually gain weight. Weight maintenance calories will depend on the weight of the person.

A 500lbs man will need to consume 4000kcals/day to not lose weight. Cutting 1800 of that is realistic and might be good advice on the LLM's part, so it really depends on how GP asked the question.

Funny thing is you can train a small BERT model to detect queries that are in categories that aren’t ready for AI “answers” with like .00000001% of the energy of an LLM.

That's (obviously) a bit of an exaggeration. BERT is just another transformer architecture. Cut down from ~100 layers to 1, ~1k dimensions to ~10, and ~10k tokens to 100, and you're only 1e6 faster / more efficient, still a factor of 10k greater than your estimate and also too small to handle the detection you're describing with any reasonable degree of accuracy.

I literally have DistilBERT models that can do this exact task in ~14ms on an NVIDIA A6000. I don’t know the precise performance per watt, but it’s really fucking low.

I use LLM to help with training data as they are great at zero shot, but after the training corpora is built a small, well trained, model will smoke an LLM in classification accuracy and are way faster - which means you can get scale and low carbon cost.

In my personal opinion there is a moral imperative to use the most efficient models possible at every step in a system design. LLM are one type of architecture and while they do a lot well, you can use a variety of energy efficient techniques to do discrete tasks much better.

Thanks for providing a concrete model to work with. Compared to GPT3.5, the number you're looking for is ~0.04%. I pointed out the napkin math because 0.00000001% was so obviously wrong even at a glance that it was hurting your claim.

And, yes, purpose-built models definitely have their place even with the advent of LLMs. I'm happy to see more people working on that sort of thing.

I applaud you doing the math! Proves you aren’t an LLM :-D

> Which is obviously dangerous advice.

Same advice as my trainer gives me.

Did they say:

  reduce their daily calorie intake to 1,500 - 1,800 calories
  or
  reduce their daily calorie intake by 1,500 - 1,800 calories

These are very different answers, unless you’re consuming ~3,300 calories per day. These kinds of ‘subtle’ phrasing issue often results in AI mistake as both words are commonly used in advice but the context is really important.

Oh yeah! No, reduce to not reduce by. Though at the time I was eating a few things that had high calories that I didn’t realize so it would have been the same.

Your trainer advises you to reduce your calorie intake to between 200 and 500 calories per day? [0] That sounds very, very hazardous for anything other than very short term use, and (given the body's inbuilt "starvation mode") probably counterproductive, even then.

[0] Note that the robot suggested to reduce calorie intake by 1,500->1,800 calories, and the recommended calorie intake is 2,000.

People losing weight are probably eating more than 2000 per day to begin with. But if you go from 2800 down to 1500 you’re already likely to exceed 3 lbs of weight loss per week that is recommended without doctor supervision. If you need to lose more than 150 lbs in a year because you’re well past morbid obesity then you need staff, not just a food plan.

If you’re eating out at Chilies, you could easily be eating 3000 calories per meal.

I recall when Krispy Kreme came out with a donut shake that was 1800 calories for the large size. It’s crazy out there.

It’s not even advice, and it’s not wrong.

I think that would explain why you’re starving, not how you’re not losing weight.