Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone

A developer has created a Swift library called Swiftlet that enables running large language models on low-end devices. This is significant because it could make AI more accessible to a wider range of users. The library has been demonstrated to run an 80B Qwen model on a Mac with 4.3 GB of RAM and a 35B model on an iPhone. Engineers can use this library to explore AI applications on resource-constrained devices. This could lead to new use cases and innovations.

Source →
FeedLens — Signal over noise Last 7 days