Tensor API¶
The Tensor class is the fundamental data structure in MLLM for representing multi-dimensional arrays. It provides a comprehensive set of operations for manipulating tensor data across different devices and data types.
#include "mllm/core/Tensor.hpp"
Creating Tensors¶
empty¶
-
static Tensor Tensor::empty(const std::vector<int32_t> &shape, DataTypes dtype = kFloat32, DeviceTypes device = kCPU)¶
Creates an uninitialized tensor with specified shape and attributes.
- Parameters:
shape – Dimensions of the tensor
dtype – Data type (default: kFloat32)
device – Target device (default: kCPU)
- Returns:
New tensor but NO MEMORY ALLOCATED!!!
zeros¶
-
static Tensor Tensor::zeros(const std::vector<int32_t> &shape, DataTypes dtype = kFloat32, DeviceTypes device = kCPU)¶
Creates a tensor filled with zeros.
- Parameters:
shape – Dimensions of the tensor
dtype – Data type (default: kFloat32)
device – Target device (default: kCPU)
- Returns:
New tensor with initialized zero values
ones¶
-
static Tensor Tensor::ones(const std::vector<int32_t> &shape, DataTypes dtype = kFloat32, DeviceTypes device = kCPU)¶
Creates a tensor filled with ones.
- Parameters:
shape – Dimensions of the tensor
dtype – Data type (default: kFloat32)
device – Target device (default: kCPU)
- Returns:
New tensor with initialized one values
arange¶
-
static Tensor Tensor::arange(float start, float end, float step, DataTypes dtype = kFloat32, DeviceTypes device = kCPU)¶
Creates a tensor with evenly spaced values within a specified range.
- Parameters:
start – Starting value
end – Ending value
step – Step size
dtype – Data type (default: kFloat32)
device – Target device (default: kCPU)
- Returns:
Tensor with evenly spaced values
random¶
-
static Tensor Tensor::random(const std::vector<int32_t> &shape, float start = -1.f, float end = 1.f, DataTypes dtype = kFloat32, DeviceTypes device = kCPU)¶
Creates a tensor with random values within a specified range.
- Parameters:
shape – Dimensions of the tensor
start – Minimum random value (default: -1.f)
end – Maximum random value (default: 1.f)
dtype – Data type (default: kFloat32)
device – Target device (default: kCPU)
- Returns:
Tensor with random values
fromVector¶
-
template<typename T>
static Tensor Tensor::fromVector(const std::vector<T> &vec, const shape_t &shape, DataTypes dtype = kFloat32, DeviceTypes device = kCPU)¶ Creates a tensor from a std::vector.
- Parameters:
vec – Source vector
shape – Dimensions of the tensor
dtype – Data type (default: kFloat32)
device – Target device (default: kCPU)
- Returns:
Tensor containing the vector data
Tensor Properties¶
name¶
-
std::string Tensor::name() const¶
Gets the tensor’s name.
- Returns:
Name string (empty if unnamed)
dtype¶
-
DataTypes Tensor::dtype() const¶
Gets data type.
- Returns:
Current data type
device¶
-
DeviceTypes Tensor::device() const¶
Gets device location.
- Returns:
Current device type
shape¶
-
shape_t Tensor::shape() const¶
Gets tensor dimensions.
- Returns:
Shape vector
stride¶
-
stride_t Tensor::stride() const¶
Gets tensor strides.
- Returns:
Stride vector
numel¶
-
size_t Tensor::numel() const¶
Calculates total number of elements.
- Returns:
Product of all dimensions
uuid¶
-
uint32_t Tensor::uuid() const¶
Gets unique tensor ID.
- Returns:
Universally unique identifier
bytes¶
-
size_t Tensor::bytes() const¶
Return how many bytes this tensor allocated.
- Returns:
Size in bytes
Memory Operations¶
alloc¶
-
Tensor &Tensor::alloc()¶
Allocates memory for a Tensor. Normally used after Tensor::empty(…).
- Returns:
Reference to this tensor for chaining
isContiguous¶
-
bool Tensor::isContiguous() const¶
Checks memory layout contiguity.
- Returns:
True if memory is contiguous
isContiguousN¶
-
bool Tensor::isContiguousN(int n) const¶
Checks memory layout contiguity for the last n dimensions.
- Parameters:
n – Number of last dimensions to check
- Returns:
True if memory is contiguous for the last n dimensions
contiguous¶
-
Tensor Tensor::contiguous()¶
Creates contiguous copy if non-contiguous.
- Returns:
Contiguous tensor (may be a view or copy)
clone¶
-
Tensor Tensor::clone()¶
Clone a tensor.
- Returns:
A copy of the tensor
copy2¶
-
void Tensor::copy2(const Tensor &src)¶
Copy data from another tensor.
- Parameters:
src – Source tensor
reshape¶
-
Tensor Tensor::reshape(const shape_t &shape)¶
Reshapes tensor without changing data order.
- Parameters:
shape – New dimensions
- Returns:
Reshaped tensor view
view¶
-
Tensor Tensor::view(const shape_t &indicies)¶
Experimental: Creates tensor view with custom indexing.
- Parameters:
indicies – View specification
- Returns:
New tensor view
- Warning:
This function is in an early age
permute¶
-
Tensor Tensor::permute(const shape_t &indices)¶
Permute tensor to a new shape.
- Parameters:
indices – New axis order
- Returns:
Permuted tensor
transpose¶
-
Tensor Tensor::transpose(int dim0, int dim1)¶
Swaps two dimensions of the tensor.
- Parameters:
dim0 – First dimension index
dim1 – Second dimension index
- Returns:
New tensor with transposed dimensions
T¶
-
Tensor Tensor::T()¶
Transpose Tensor at last 2 dims.
- Returns:
Transposed tensor
repeat¶
-
Tensor Tensor::repeat(int32_t multiplier, int32_t dim)¶
Repeats tensor along a dimension.
- Parameters:
multiplier – Repeat multiplier
dim – Dimension to repeat along
- Returns:
Repeated tensor
unsqueeze¶
-
Tensor Tensor::unsqueeze(int32_t dim)¶
Unsqueeze tensor along a dimension.
- Parameters:
dim – Dimension to unsqueeze
- Returns:
Unsqueezed tensor
to (device)¶
-
Tensor Tensor::to(DeviceTypes device)¶
Transfers tensor to specified device.
- Parameters:
device – Target device
- Returns:
New tensor on target device (data copied if needed)
to (dtype)¶
-
Tensor Tensor::to(DataTypes dtype)¶
Converts tensor to specified data type.
- Parameters:
dtype – Target data type
- Returns:
New tensor with converted data type
cpu¶
-
Tensor Tensor::cpu()¶
Shortcut for moving tensor to CPU.
- Returns:
CPU-resident tensor
cuda¶
-
Tensor Tensor::cuda()¶
Shortcut for moving tensor to GPU.
- Returns:
GPU-resident tensor
Element-wise Arithmetic Operations¶
operator+¶
-
Tensor Tensor::operator+(const Tensor &rhs)¶
Element-wise addition with another tensor.
- Parameters:
rhs – Right-hand side tensor
- Returns:
Result tensor
-
Tensor Tensor::operator+(float rhs)¶
Element-wise addition with a scalar.
- Parameters:
rhs – Right-hand side scalar value
- Returns:
Result tensor
operator-¶
-
Tensor Tensor::operator-(const Tensor &rhs)¶
Element-wise subtraction with another tensor.
- Parameters:
rhs – Right-hand side tensor
- Returns:
Result tensor
-
Tensor Tensor::operator-(float rhs)¶
Element-wise subtraction with a scalar.
- Parameters:
rhs – Right-hand side scalar value
- Returns:
Result tensor
-
Tensor Tensor::operator-()¶
Negative.
- Returns:
Negated tensor
operator*¶
-
Tensor Tensor::operator*(const Tensor &rhs)¶
Element-wise multiplication with another tensor.
- Parameters:
rhs – Right-hand side tensor
- Returns:
Result tensor
-
Tensor Tensor::operator*(float rhs)¶
Element-wise multiplication with a scalar.
- Parameters:
rhs – Right-hand side scalar value
- Returns:
Result tensor
operator/¶
-
Tensor Tensor::operator/(const Tensor &rhs)¶
Element-wise division with another tensor.
- Parameters:
rhs – Right-hand side tensor
- Returns:
Result tensor
-
Tensor Tensor::operator/(float rhs)¶
Element-wise division with a scalar.
- Parameters:
rhs – Right-hand side scalar value
- Returns:
Result tensor
abs¶
-
Tensor Tensor::abs()¶
Computes the absolute value of the tensor elements.
- Returns:
Tensor with absolute values
clip¶
-
Tensor Tensor::clip(float min_val, float max_val)¶
Clips (limits) the values in a tensor.
- Parameters:
min_val – Minimum value
max_val – Maximum value
- Returns:
A tensor with clipped values
Reduction Operations¶
topk¶
-
std::array<Tensor, 2> Tensor::topk(int32_t k, int32_t dim = -1, bool largest = true, bool sorted = true)¶
Finds the top k largest (or smallest) elements in a tensor.
- Parameters:
k – Number of top elements to find
dim – Dimension along which to find top k elements (default: -1)
largest – If true, find the largest elements; otherwise, find the smallest (default: true)
sorted – If true, the result will be sorted by value (default: true)
- Returns:
An array containing values and indices of the top k elements
min¶
-
Tensor Tensor::min(bool keep_dim = false, int32_t dim = 0x7fffffff)¶
Get minimum values.
- Parameters:
keep_dim – If true, keep the reduced dimension (default: false)
dim – Dimension to reduce. If 0x7fffffff, return a scalar value (default: 0x7fffffff)
- Returns:
Tensor with minimum values
max¶
-
Tensor Tensor::max(bool keep_dim = false, int32_t dim = 0x7fffffff)¶
Get maximum values.
- Parameters:
keep_dim – If true, keep the reduced dimension (default: false)
dim – Dimension to reduce. If 0x7fffffff, return a scalar value (default: 0x7fffffff)
- Returns:
Tensor with maximum values
sum¶
-
Tensor Tensor::sum(bool keep_dim = false, int32_t dim = 0x7fffffff)¶
Get sum of elements.
- Parameters:
keep_dim – If true, keep the reduced dimension (default: false)
dim – Dimension to reduce. If 0x7fffffff, return a scalar value (default: 0x7fffffff)
- Returns:
Tensor with sum of elements
mean¶
-
Tensor Tensor::mean(bool keep_dim = false, int32_t dim = 0x7fffffff)¶
Get mean of elements.
- Parameters:
keep_dim – If true, keep the reduced dimension (default: false)
dim – Dimension to reduce. If 0x7fffffff, return a scalar value (default: 0x7fffffff)
- Returns:
Tensor with mean of elements
Indexing and Slicing¶
operator[]¶
-
Tensor Tensor::operator[](const SliceIndices &slice_index) const¶
Creates a shallow view (slice) of the tensor.
- Parameters:
slice_index – Slice specification
- Returns:
New tensor view referencing the sliced data
- Note:
Uses shallow copy when step size is 1; may be unsafe for GPU tensors
-
Tensor Tensor::operator[](const ComplexIndexingList &complex_indexing) const¶
Creates a deep copy of the tensor with complex indexing.
- Parameters:
complex_indexing – Complex indexing specification
- Returns:
New tensor with indexed data
setName¶
-
Tensor &Tensor::setName(const std::string &name)¶
Sets tensor name.
- Parameters:
name – New name for tensor
- Returns:
Reference to this tensor for chaining
setMemType¶
-
Tensor &Tensor::setMemType(TensorMemTypes mem_type)¶
Sets memory type.
- Parameters:
mem_type – New memory type
- Returns:
Reference to this tensor for chaining
memType¶
-
TensorMemTypes Tensor::memType() const¶
Gets memory type.
- Returns:
Memory type identifier
Auxiliary Tensor Views¶
allocExtraTensorView¶
-
Tensor &Tensor::allocExtraTensorView(const std::string &extra_tensor_name, const std::vector<int32_t> &shape, DataTypes dtype = kFloat32, DeviceTypes device = kCPU)¶
Creates and attaches an auxiliary tensor view to this tensor.
- Parameters:
extra_tensor_name – Unique identifier for the auxiliary view
shape – Dimensions of the auxiliary tensor
dtype – Data type (default: kFloat32)
device – Target device (default: kCPU)
- Returns:
Reference to this tensor for chaining
- Note:
This function is designed for quantized Tensor. If one Tensor is quantized to int8 using per tensor quantization method, you can use this_tensor.allocExtraTensorView(“scale”, shape, kFloat32, kCPU); to attach a scale tensor to this tensor.
getExtraTensorViewInTensor¶
-
Tensor Tensor::getExtraTensorViewInTensor(const std::string &extra_tensor_name)¶
Retrieves a previously attached auxiliary tensor view.
- Parameters:
extra_tensor_name – Name of the auxiliary tensor
- Returns:
The requested tensor view
- Note:
This function is designed for quantized Tensor. If one Tensor is quantized to int8 using per tensor quantization method, you can use this_tensor.getExtraTensorViewInTensor(“scale”).item<float>(); to get a scale tensor from this tensor.
Utility Functions¶
isNil¶
-
bool Tensor::isNil() const¶
Check if this tensor is not initialized.
- Returns:
true if tensor is nil, false otherwise
nil¶
-
static Tensor Tensor::nil()¶
Create a nil tensor.
- Returns:
Nil tensor
operator bool¶
-
explicit Tensor::operator bool() const noexcept¶
Check if this tensor is initialized.
- Returns:
true if tensor is initialized, false otherwise
delete¶
-
void Tensor::delete_() noexcept¶
Delete tensor resources.
operator delete¶
-
void Tensor::operator delete(void *ptr) noexcept¶
Custom delete operator.