Python PyTorch: How to Access the Metadata of a Tensor in PyTorch

Learn how to inspect PyTorch tensor metadata like shape, dtype, device, strides, and memory layout to debug, optimize performance, and ensure compatibility between operations.

When working with PyTorch tensors, understanding their metadata - such as shape, data type, device, and memory layout - is essential for debugging, ensuring compatibility between operations, and optimizing performance. Metadata tells you everything about a tensor's structure without looking at the actual data.

This guide covers all the key metadata properties and methods available on PyTorch tensors, with practical examples.

What Is Tensor Metadata?

Tensor metadata is information about the tensor's structure rather than its values. This includes:

Shape / Size - the dimensions of the tensor
Data type (dtype) - whether it holds floats, integers, etc.
Device - whether it's on CPU or GPU
Number of elements - total count of values
Number of dimensions - how many axes the tensor has
Memory layout - how the data is stored in memory

Shape and Size

The most commonly accessed metadata is a tensor's shape - its dimensions. PyTorch provides two equivalent ways to get this:

import torch

tensor = torch.tensor([
    [5, 1, 7],
    [7, 2, 9],
    [4, 7, 9],
    [8, 12, 14],
    [2, 4, 7]
])

print("Using .size():", tensor.size())
print("Using .shape: ", tensor.shape)

Output:

Using .size(): torch.Size([5, 3])
Using .shape:  torch.Size([5, 3])

.size() is a method (can accept a dimension argument).
.shape is a property (more concise, same as NumPy).

Getting the Size of a Specific Dimension

import torch

tensor = torch.randn(3, 4, 5)

print("Full shape:", tensor.shape)
print("Dimension 0:", tensor.size(0))  # 3
print("Dimension 1:", tensor.size(1))  # 4
print("Dimension 2:", tensor.size(2))  # 5

Output:

Full shape: torch.Size([3, 4, 5])
Dimension 0: 3
Dimension 1: 4
Dimension 2: 5

Number of Dimensions

The ndim property (or dim() method) returns how many axes the tensor has:

import torch

scalar = torch.tensor(42)
vector = torch.tensor([1, 2, 3])
matrix = torch.tensor([[1, 2], [3, 4]])
tensor_3d = torch.randn(2, 3, 4)

print(f"Scalar dimensions: {scalar.ndim}")
print(f"Vector dimensions: {vector.ndim}")
print(f"Matrix dimensions: {matrix.ndim}")
print(f"3D tensor dimensions: {tensor_3d.dim()}")

Output:

Scalar dimensions: 0
Vector dimensions: 1
Matrix dimensions: 2
3D tensor dimensions: 3

Total Number of Elements

Use torch.numel() or .numel() to count the total number of elements in a tensor:

import torch

tensor = torch.tensor([
    [5, 1, 7],
    [7, 2, 9],
    [4, 7, 9],
    [8, 12, 14],
    [2, 4, 7]
])

print("Shape:", tensor.shape)
print("Total elements:", torch.numel(tensor))
print("Total elements:", tensor.numel())  # Equivalent

Output:

Shape: torch.Size([5, 3])
Total elements: 15
Total elements: 15

The total element count is the product of all dimensions: 5 × 3 = 15.

Data Type (dtype)

Every tensor has a data type that determines what kind of values it stores:

import torch

int_tensor = torch.tensor([1, 2, 3])
float_tensor = torch.tensor([1.0, 2.0, 3.0])
bool_tensor = torch.tensor([True, False, True])
explicit_tensor = torch.zeros(3, dtype=torch.float16)

print("Integer tensor dtype:", int_tensor.dtype)
print("Float tensor dtype:", float_tensor.dtype)
print("Boolean tensor dtype:", bool_tensor.dtype)
print("Explicit dtype:", explicit_tensor.dtype)

Output:

Integer tensor dtype: torch.int64
Float tensor dtype: torch.float32
Boolean tensor dtype: torch.bool
Explicit dtype: torch.float16

Common PyTorch data types

dtype	Description	Common Use
`torch.float32`	32-bit float (default for floats)	Neural network weights
`torch.float16`	16-bit float	Mixed precision training
`torch.int64`	64-bit integer (default for ints)	Indices, labels
`torch.bool`	Boolean	Masks, conditions

Device (CPU vs GPU)

The .device property tells you where the tensor is stored - CPU or a specific GPU:

import torch

cpu_tensor = torch.tensor([1, 2, 3])
print("Device:", cpu_tensor.device)

if torch.cuda.is_available():
    gpu_tensor = cpu_tensor.to('cuda')
    print("GPU Device:", gpu_tensor.device)
else:
    print("CUDA not available, tensor stays on CPU")

Output:

Device: cpu
CUDA not available, tensor stays on CPU

Device mismatch errors

Operations between tensors on different devices will raise a RuntimeError. Always check devices before performing operations:

result = cpu_tensor + gpu_tensor  # RuntimeError!

result = cpu_tensor.to('cuda') + gpu_tensor

Memory Layout and Strides

Strides describe how many elements to skip in memory to move to the next position along each dimension. This metadata is important for understanding memory efficiency and contiguity:

import torch

tensor = torch.tensor([
    [1, 2, 3],
    [4, 5, 6]
])

print("Shape:", tensor.shape)
print("Strides:", tensor.stride())
print("Is contiguous:", tensor.is_contiguous())

Output:

Shape: torch.Size([2, 3])
Strides: (3, 1)
Is contiguous: True

The stride (3, 1) means: moving to the next row requires skipping 3 elements, and moving to the next column requires skipping 1 element.

After Transposing

import torch

tensor = torch.tensor([[1, 2, 3], [4, 5, 6]])
transposed = tensor.T

print("Transposed shape:", transposed.shape)
print("Transposed strides:", transposed.stride())
print("Is contiguous:", transposed.is_contiguous())

Output:

Transposed shape: torch.Size([3, 2])
Transposed strides: (1, 3)
Is contiguous: False

After transposing, the tensor is no longer contiguous in memory. Some operations require contiguous tensors, which you can create with .contiguous().

Requires Grad (Gradient Tracking)

The requires_grad property indicates whether PyTorch tracks operations on this tensor for automatic differentiation (essential for training neural networks):

import torch

data = torch.tensor([1.0, 2.0, 3.0])
print("Requires grad:", data.requires_grad)

weights = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
print("Requires grad:", weights.requires_grad)
print("Grad function:", weights.grad_fn)

Output:

Requires grad: False
Requires grad: True
Grad function: None

Complete Metadata Summary Function

Here's a utility function that prints all metadata for any tensor:

import torch

def tensor_info(tensor, name="Tensor"):
    """Print comprehensive metadata about a tensor."""
    print(f"--- {name} ---")
    print(f"  Shape:         {tensor.shape}")
    print(f"  Dimensions:    {tensor.ndim}")
    print(f"  Total elements:{tensor.numel()}")
    print(f"  Data type:     {tensor.dtype}")
    print(f"  Device:        {tensor.device}")
    print(f"  Strides:       {tensor.stride()}")
    print(f"  Contiguous:    {tensor.is_contiguous()}")
    print(f"  Requires grad: {tensor.requires_grad}")
    print()

tensor_info(torch.randn(3, 4, 5), "3D Random Tensor")
tensor_info(torch.zeros(10, dtype=torch.int32), "Integer Zeros")
tensor_info(torch.ones(2, 2, requires_grad=True), "Trainable Weights")

Output:

--- 3D Random Tensor ---
  Shape:         torch.Size([3, 4, 5])
  Dimensions:    3
  Total elements:60
  Data type:     torch.float32
  Device:        cpu
  Strides:       (20, 5, 1)
  Contiguous:    True
  Requires grad: False

--- Integer Zeros ---
  Shape:         torch.Size([10])
  Dimensions:    1
  Total elements:10
  Data type:     torch.int32
  Device:        cpu
  Strides:       (1,)
  Contiguous:    True
  Requires grad: False

--- Trainable Weights ---
  Shape:         torch.Size([2, 2])
  Dimensions:    2
  Total elements:4
  Data type:     torch.float32
  Device:        cpu
  Strides:       (2, 1)
  Contiguous:    True
  Requires grad: True

Quick Reference

Property/Method	Returns	Description
`.shape`	`torch.Size`	Dimensions of the tensor
`.size()`	`torch.Size`	Same as `.shape` (can take dim argument)
`.ndim` / `.dim()`	`int`	Number of dimensions
`.numel()`	`int`	Total number of elements
`.dtype`	`torch.dtype`	Data type of elements
`.device`	`torch.device`	CPU or GPU location
`.stride()`	`tuple`	Memory strides per dimension
`.is_contiguous()`	`bool`	Whether memory layout is contiguous
`.requires_grad`	`bool`	Whether gradients are tracked
`.grad_fn`	`Function` or `None`	Gradient function (if part of computation graph)

Conclusion

Accessing tensor metadata in PyTorch is straightforward and essential for building reliable deep learning pipelines.

Use .shape and .dtype for everyday checks,
Use .device to ensure tensors are on the correct hardware
Use .stride() and .is_contiguous() when optimizing memory layout.

Understanding these properties helps you debug shape mismatches, prevent device errors, and write more efficient PyTorch code.