• lime!@feddit.nu
    link
    fedilink
    English
    arrow-up
    1
    ·
    9 hours ago

    llamafile is not really “effective”. it’s incredibly impressive, but it’s the opposite of effective. it’s a collection of a bunch of hacks reliant on coincidences in OS design, and works by basically recompiling itself on the fly to work with different architectures.

    if you want effective, run llama.cpp compiled with actual optimizations for your platform.