Get Started

mllm is an open-source project and you can freely use it in your projects. You can find the source code on GitHub. The project is also developed and maintained publicly. Check out the Contributing Guide to see how you can help.

Contributing Guide Learn how to contribute to mllm.

Before you start

1. Clone the repository

git clone https://github.com/UbiquitousLearning/mllm

2. Check prerequisites Although mllm is a standalone library, it requires some tools to build the project and some other libs to accelerate the inference.

• gcc(11.4+) / clang (11.0+)
• CMake >= 3.18
• OpenMP Libs.
• Android NDK Toolchains >= 26

Build the project

Build Target

Currently, mllm can run on Android, Linux, and macOS with architectures of x86_64, and arm64.

mllm offers two forms of user interface: Command Line Interface Demo and an Android Demo App .

Build For CLI Demo

CLI Apps are built by default. You can find the executable file in the bin or bin-arm directory.

Tip

Please note, that before executing the binary file, you need to place the model files and Tokenizer in the models directory under the current directory.

• To Build for Host OS:

cd scripts && ./build.sh

• To Build for Android: (NDK is required)

export ANDROID_NDK=/path/to/your/ndk
cd scripts && ./build_android.sh

The binary demo application should be located in the bin or bin-arm directory. If you need to test the command line demo on the Android platform, you can use the adb push command to push the binary file to the device, and run it with the adb shell command.

Tip

For the most commonly used LLAMA model, we provide a run script to use adb to quickly test the performance of mllm. Before running the script, you need to push the model files and Tokenizer vocab to the /data/local/tmp/mllm/models directory on the device.

cd scripts && ./run_llama.sh

Build For Android Demo App

Android Demo App is currently in a very early stage, and it is maintained in a separate repository. You can find it in the android sub-module.

Tip

Please note, when using the Android Demo App, you need to push the model files and Tokenizer to the /sdcard/Download/model directory on the device before launching the App.

For more details, check the README.md file in the android directory or visit the Chatbot Repository.