Whenver someone suggest that companies should just simply selfhost LLMs on their machines because they have tried it at ...

Whenver someone suggest that companies should just simply selfhost LLMs on their machines because they have tried it at home and it works.I notice what everyone expects or takes for granted:- the latest and biggest model- availability (redundancy anyone?)- high response time (okay, in a company with hundreds or thousands of users, you have to share the hardware)- freedom to choose arbitrary models (okay, how many can you run in parallel in one machine?)- be cheaper than hyperscalersAnd having people maintaing it all ..#WorksOnMyMachine #llm #genai

Read Original

Related