I’ve tried coding and every one I’ve tried fails unless really, really basic small functions like what you learn as a newbie compared to say 4o mini that can spit out more sensible stuff that works.

I’ve tried explanations and they just regurgitate sentences that can be irrelevant, wrong, or get stuck in a loop.

So. what can I actually use a small LLM for? Which ones? I ask because I have an old laptop and the GPU can’t really handle anything above 4B in a timely manner. 8B is about 1 t/s!

    • CrayonDevourer@lemmy.worldEnglish
      21·
      17 hours ago

      Yes. The small LLM isn’t retrieving data, it’s just understanding context of text enough to know what “Facts” need to be written to a file. I’m using the publicly released Deepseek models from a couple of months ago.

      • catty@lemmy.worldOPEnglish
        1·
        2 hours ago

        Some questions and because you don’t actually understand, also, the answers.

        • what does the LLM understand the context of, (other user’s data owned by Twitch)
        • How is the LLM fed that data? (You store it and feed it to the LLM)
        • Do you use Twitch’s data and its users data through an AI without their consent? (Most likely, yes)
        • Do you have consent from the users to store ‘facts’ about them (You’re pissy, so obviously not)
        • Are you then storing that processed data? (Yes, you are, written to a file)
        • Is the purpose this data processing commercial (Yes, it is, designed to increase viewer count for the user of this system - and before you retort “OMG it helps twitch too”… Uhm no, Twitch has the viewers if not watching him, watching someone else)

        I mean yeah, it’s a use case, but own up to the fact that you’re wrong. Or be pissy. I don’t care.