localai - Tech blog

Tech blog

localai

A collection of 5 posts

What Can You Actually Do With a Local LLM?

What Can You Actually Do With a Local LLM?

Everyone talks about ChatGPT, Claude, and Gemini. But a growing number of people are running large language models entirely on their own hardware. No API keys. No subscriptions. No data leaving their machine. The question is simple: what can you realistically do with a local LLM? More than you think.

Will Self-Hosted Local AI Go Mainstream, or Stay a Niche for Nerds?

Will Self-Hosted Local AI Go Mainstream, or Stay a Niche for Nerds?

A recent thread on r/SelfHostedAI sparked the question: self-hosted AI won't take off if the average user has to fight for it. Is that right? The post that started the debate The premise of the r/SelfHostedAI discussion was simple and uncomfortable: "Self-hosted AI

Translating EPUB Ebooks with Local LLM and for FREE. An Effective Approach

Translating EPUB Ebooks with Local LLM and for FREE. An Effective Approach

Translating an entire ebook is more than just converting text; it involves handling complex structures like formatting, code, images, and managing book length. This guide explores how to achieve this for free using local Large Language Models (LLMs). Challenges in Ebook Translation Directly feeding an entire EPUB to an LLM

llama.cpp benchmarks on AMD Ryzen 7 7700

Inspired by this reddit post, here my results # HIP_VISIBLE_DEVICES=1 llama-bench --model /models/Qwen3-4B-IQ4_NL.gguf -ngl 99 --flash-attn --no-mmap ggml_cuda_init: failed to initialize ROCm: no ROCm-capable device is detected | model | size | params | backend | ngl | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: | | qwen3 4B

L'Intelligenza Artificiale Locale su Linux: Sfide e Opportunità

L'adozione dell'intelligenza artificiale (IA) locale su sistemi Linux presenta ancora diverse sfide, nonostante i progressi tecnologici degli ultimi anni. Questo articolo esplora le problematiche attuali e le considerazioni da fare per chi è interessato a utilizzare l'IA locale su Linux. Architetture e Compatibilità 1.