Python PyTorch: How to Create and Initialize Tensors in Python with PyTorch

How to create and initialize PyTorch tensors using methods like torch.tensor(), torch.zeros(), and torch.randn(), with tips for choosing the right dtype, device, and memory-efficient approach.

Tensors are the fundamental data structures in PyTorch, serving as the building blocks for all deep learning operations. Similar to NumPy arrays but with GPU acceleration and automatic differentiation support, tensors store everything from input data to model parameters. Mastering tensor creation and initialization is essential for building efficient neural networks and implementing custom deep learning solutions.

Creating Tensors from Existing Data

Convert Python lists or NumPy arrays into PyTorch tensors using torch.tensor():

import torch
import numpy as np

vector = torch.tensor([1.0, 2.0, 3.0, 4.0])
print(f"Vector: {vector}")
print(f"Shape: {vector.shape}, Dtype: {vector.dtype}")

matrix = torch.tensor([[1, 2, 3], [4, 5, 6]])
print(f"\nMatrix:\n{matrix}")
print(f"Shape: {matrix.shape}")

np_array = np.array([[1.0, 2.0], [3.0, 4.0]])
from_numpy = torch.from_numpy(np_array)
print(f"\nFrom NumPy:\n{from_numpy}")

Output:

Vector: tensor([1., 2., 3., 4.])
Shape: torch.Size([4]), Dtype: torch.float32

Matrix:
tensor([[1, 2, 3],
        [4, 5, 6]])
Shape: torch.Size([2, 3])

From NumPy:
tensor([[1., 2.],
        [3., 4.]], dtype=torch.float64)

Initializing with Constant Values

Create tensors filled with specific values for initializing biases, masks, or placeholders:

import torch

zeros = torch.zeros(3, 4)
print(f"Zeros (3x4):\n{zeros}")

ones = torch.ones(2, 3)
print(f"\nOnes (2x3):\n{ones}")

filled = torch.full((2, 2), fill_value=3.14)
print(f"\nFilled with 3.14:\n{filled}")

identity = torch.eye(3)
print(f"\nIdentity (3x3):\n{identity}")

Output:

Zeros (3x4):
tensor([[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]])

Ones (2x3):
tensor([[1., 1., 1.],
        [1., 1., 1.]])

Filled with 3.14:
tensor([[3.1400, 3.1400],
        [3.1400, 3.1400]])

Identity (3x3):
tensor([[1., 0., 0.],
        [0., 1., 0.],
        [0., 0., 1.]])

Random Tensor Initialization

Random initialization is critical for neural network weights. PyTorch provides various distributions:

import torch

uniform = torch.rand(3, 3)
print(f"Uniform [0,1):\n{uniform}")

normal = torch.randn(3, 3)
print(f"\nNormal (μ=0, σ=1):\n{normal}")

integers = torch.randint(low=0, high=10, size=(2, 3))
print(f"\nRandom integers [0,10):\n{integers}")

custom_normal = torch.normal(mean=5.0, std=2.0, size=(2, 2))
print(f"\nCustom normal (μ=5, σ=2):\n{custom_normal}")

Output:

Uniform [0,1):
tensor([[0.8427, 0.2165, 0.8221],
        [0.6094, 0.9972, 0.6888],
        [0.4389, 0.0244, 0.7633]])

Normal (μ=0, σ=1):
tensor([[-1.1526, -1.1820,  0.5721],
        [ 0.3129,  0.2032,  0.9088],
        [-0.2895, -0.7636, -0.5447]])

Random integers [0,10):
tensor([[3, 8, 8],
        [2, 0, 9]])

Custom normal (μ=5, σ=2):
tensor([[2.6919, 2.0961],
        [4.1802, 1.8831]])

Sequential and Range Tensors

Create tensors with sequential or evenly spaced values:

import torch

range_tensor = torch.arange(0, 10, 2)
print(f"Arange (0 to 10, step 2): {range_tensor}")

linspace = torch.linspace(0, 1, steps=5)
print(f"Linspace (0 to 1, 5 steps): {linspace}")

logspace = torch.logspace(0, 2, steps=3)  # 10^0 to 10^2
print(f"Logspace: {logspace}")

Output:

Arange (0 to 10, step 2): tensor([0, 2, 4, 6, 8])
Linspace (0 to 1, 5 steps): tensor([0.0000, 0.2500, 0.5000, 0.7500, 1.0000])
Logspace: tensor([  1.,  10., 100.])

Specifying Data Types and Device

Control tensor precision and hardware placement:

import torch

float32_tensor = torch.zeros(2, 2, dtype=torch.float32)
float64_tensor = torch.zeros(2, 2, dtype=torch.float64)
int_tensor = torch.zeros(2, 2, dtype=torch.int64)

print(f"Float32: {float32_tensor.dtype}")
print(f"Float64: {float64_tensor.dtype}")
print(f"Int64: {int_tensor.dtype}")

device = "cuda" if torch.cuda.is_available() else "cpu"
gpu_tensor = torch.randn(3, 3, device=device)
print(f"\nDevice: {gpu_tensor.device}")

cpu_tensor = torch.randn(2, 2)
gpu_tensor = cpu_tensor.to(device)

Output:

Float32: torch.float32
Float64: torch.float64
Int64: torch.int64

Device: cpu

Creating Tensors Like Existing Ones

Create new tensors matching the shape, dtype, or device of existing tensors:

import torch

original = torch.randn(3, 4, dtype=torch.float32, device="cpu")

zeros_like = torch.zeros_like(original)

ones_like = torch.ones_like(original)

rand_like = torch.rand_like(original)

new_tensor = torch.empty_like(original).fill_(5.0)

print(f"Original shape: {original.shape}")
print(f"All match dtype ({zeros_like.dtype}) and device ({zeros_like.device})")

Output:

Original shape: torch.Size([3, 4])
All match dtype (torch.float32) and device (cpu)

Quick Reference

Method	Description	Typical Use Case
`torch.tensor()`	From existing data	Loading datasets
`torch.zeros()`	All zeros	Bias initialization
`torch.ones()`	All ones	Masks, scaling
`torch.randn()`	Normal distribution	Weight initialization
`torch.rand()`	Uniform [0, 1)	Dropout masks
`torch.arange()`	Sequential integers	Positional encoding
`torch.eye()`	Identity matrix	Linear algebra
`torch.empty()`	Uninitialized	Performance-critical allocation

Mastering these tensor creation methods provides the foundation for implementing efficient PyTorch models, from simple linear layers to complex transformer architectures.