Get Started

mllm is an open-source project and you can freely use it in your projects. You can find the source code on GitHub. The project is also developed and maintained publicly. Check out the Contributing Guide to see how you can help.

Contributing Guide Learn how to contribute to mllm.

Before you start

1. Clone the repository

git clone https://github.com/UbiquitousLearning/mllm

2. Check prerequisites Although mllm is a standalone library, it requires some tools to build the project and some other libs to accelerate the inference.

• gcc(11.4+) / clang (11.0+)
• CMake >= 3.18
• OpenMP Libs.
• Android NDK Toolchains >= 26

Build the project

Build Target

Currently, mllm can run on Android, Linux, and macOS with architectures of x86_64, and arm64.

mllm offers two forms of user interface: Command Line Interface Demo and an Android Demo App .

Build For CLI Demo

CLI Apps are built by default. You can find the executable file in the bin or bin-arm directory.

Tip

Please note, that before executing the binary file, you need to place the model files and Tokenizer in the models directory under the current directory.

• To Build for Host OS:

cd scripts && ./build.sh

• To Build for Android: (NDK is required)

export ANDROID_NDK=/path/to/your/ndk
cd scripts && ./build_android.sh

The binary demo application should be located in the bin or bin-arm directory. If you need to test the command line demo on the Android platform, you can use the adb push command to push the binary file to the device, and run it with the adb shell command.

Tip

For the most commonly used LLAMA model, we provide a run script to use adb to quickly test the performance of mllm. Before running the script, you need to push the model files and Tokenizer vocab to the /data/local/tmp/mllm/models directory on the device.

cd scripts && ./run_llama.sh

Build For Android Demo App

Android Demo App is currently in a very early stage, and it is maintained in a separate repository. You can find it in the android sub-module.

Tip

Please note, when using the Android Demo App, you need to push the model files and Tokenizer to the /sdcard/Download/model directory on the device before launching the App.

Be sure to organize the files in the following file names:

File	FileName
Fuyu Model	fuyu.mllm
Fuyu Vocab	vocab_uni.mllm
LLaMA Model	llama_2.mllm
LLaMA Vocab	vocab.mllm

1. Fetch the sub-module

git submodule update --init --recursive # clone the android demo app

2. Now you can import the project into Android Studio and build it.
3. If you do not use Android Studio, you may need to manually set up JDK(17+) and Android SDK(30+) environment, and then build it with gradle.

• Build JNI Libs (Optional) The mllm JNI Libs Binary is provided along with the Android Demo App. You can find it in the android/app/src/main/cpp/libs directory. But if you want to build it yourself, you can follow the steps below:

export ANDROID_NDK=/path/to/your/ndk
cmake -DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK}/build/cmake/android.toolchain.cmake" -DAPK=ON -DANDROID_ABI="arm64-v8a" -DANDROID_NDK="${ANDROID_NDK}" -DANDROID_PLATFORM=android-28 -B./build_arm/ .
cmake --build ./build_arm/ --target all -- -j$(nproc)

The libmllm_lib.a should be located in the build_arm directory.

For Your Convenience, we also provide a script to build the whole App:

cd scripts && ./build_android_app.sh