Lately, PyTorch has gain a lot of popularity among the data science community, as one of the most used Deep Learning libraries.

This library is based on tensors, which can be understood as a n-dimension matrix from which you can perform mathematical operations and build Deep Learning Models.

In this article, I will introduce you to five useful tensor functions that you can use in PyTorch.

PyTorch Functions

  • torch.Tensor.expand(*sizes)
  • torch.reshape(input, shape)
  • torch.add(input, other, out=None)
  • torch.t(input)
  • torch.numel(input)

In [0]:

# Import torch
import torch

torch.Tensor.expand(*sizes)

This function allows you to set the quantity of elements per dimension.

In [11]:

x = torch.tensor([[1], [2], [3]]) # We create a 2-dimension tensor with 3 rows and 1 column
print(x.size())

print(x.expand(3, 4)) # For the expand function arguments, we use the same length for the first dimension (3), and increase the second dimension from 1 to 4

torch.Size([3, 1])
tensor([[1, 1, 1, 1],
        [2, 2, 2, 2],
        [3, 3, 3, 3]])

Here, we created a tensor “x” with 3 rows and 1 column. Then, we applied to our tensor the expand() function to increase it’s number of columns to 4. As you can see, the columns were replicated 3 times.

In [12]:

print(x.expand(-1, 4)) # -1 means not changing the size of that dimension

tensor([[1, 1, 1, 1],
        [2, 2, 2, 2],
        [3, 3, 3, 3]])

This example gives the same output, but instead of using “3” as the first argument, we used “-1” which assings the original size of that dimension.

In [13]:

print(x.expand(3, 4, 7))

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-13-0d5b196f7923> in <module>()
----> 1 print(x.expand(3, 4, 7))

RuntimeError: The expanded size of the tensor (4) must match the existing size (3) at non-singleton dimension 1.  Target sizes: [3, 4, 7].  Tensor sizes: [3, 1]

Here, we can see what happens when we try to add more dimensions to the tensor from the expand method. We get an error because that can’t be done by using that method.

This function is very useful when you want to reshape a tensor to make it compatible with other tensors for arithmetic operations. Two tensors are compatible when for each dimension pair, they are equal.

torch.reshape(input, shape)

This function returns a tensor with the same data and number of elements as input, but with the specified shape. A single dimension may be -1, in which case it’s inferred from the remaining dimensions and the number of elements in input.

In [14]:

a = torch.arange(4.) # 1-dimension tensor
print(a)
torch.reshape(a, (2, 2)) # reshape to a 2-dimension tensor

tensor([0., 1., 2., 3.])

Out[14]:

tensor([[0., 1.],
        [2., 3.]])

Here, we created a 1-dimension tensor with 4 elements. Then we used the reshape function to turn it into a 2-dimensions tensor, with 2 rows and 2 columns.

In [15]:

b = torch.tensor([[0, 1], [2, 3]]) # 2-dimensions tensor
print(b)
torch.reshape(b, (-1,)) # reshape to 1-dimension tensor

tensor([[0, 1],
        [2, 3]])

Out[15]:

tensor([0, 1, 2, 3])

In this example, we created a 2-dimensions tensor with 4 elements. Then we used the reshape function to turn it into a 1-dimension tensor.

In [16]:

a = torch.arange(4.) # 1-dimension tensor
print(a)
torch.reshape(a, (2, 3)) # We use the reshape function with an invalid shape

tensor([0., 1., 2., 3.])

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-16-456a28a19a6c> in <module>()
      1 a = torch.arange(4.) # 1-dimension tensor
      2 print(a)
----> 3 torch.reshape(a, (2, 3)) # We use the reshape function with an invalid shape

RuntimeError: shape '[2, 3]' is invalid for input of size 4

If you use a shape that is incompatible with the amount of elements of the tensor, you will get an error like this one.

As the previous function, this one is very useful when you want to reshape a tensor to make it compatible with other tensors for arithmetic operations.

torch.add(input, other, alpha=1, out=None)

Adds other (it can be an escalar or a tensor) to each element of the tensor input and returns a new resulting tensor.

When alpha is set, it works as the scalar multiplier for other.

In [17]:

a = torch.randn(4) # 1-dimension tensor
print(a)

torch.add(a, 20)

tensor([-0.1182, -0.1796,  1.7664,  0.4416])

Out[17]:

tensor([19.8818, 19.8204, 21.7664, 20.4416])

In this case, we are adding a scalar (20) to the tensor by using the add() function.

In [18]:

a = torch.randn(4)
print(a)
b = torch.randn(4, 1)
print(b)

torch.add(a, b, alpha=10)

tensor([-0.6033,  1.4754,  0.4251,  2.3262])
tensor([[ 0.6813],
        [-0.0768],
        [-1.0175],
        [ 0.1177]])

Out[18]:

tensor([[  6.2099,   8.2886,   7.2383,   9.1393],
        [ -1.3715,   0.7072,  -0.3431,   1.5580],
        [-10.7783,  -8.6996,  -9.7499,  -7.8489],
        [  0.5734,   2.6521,   1.6018,   3.5028]])

In this example, we are adding the tensor a with the product of the scalar alpha times the tensor b.

In [19]:

a = torch.randn(4)
print(a)

torch.add(a, (20, 25, 30, 35)) # You can not add a tensor with a tuple

tensor([ 1.1978,  0.6394, -1.5532, -1.4733])

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-19-33cc2e2ea12f> in <module>()
      2 print(a)
      3 
----> 4 torch.add(a, (20, 25, 30, 35)) # You can not add a tensor with a tuple

TypeError: add(): argument 'other' (position 2) must be Tensor, not tuple

When you are doing arithmetic operations with tensors, make sure to do it with scalars or tensors. In this case, we can see that we get an error when we try to add a tensor with a tuple.

This is one of the functions that we will use when we want to do arithmetic operations over tensors.

torch.t(input)

Expects input to be <= 2-D tensor and transposes dimensions 0 and 1.

0-D and 1-D tensors are returned as is. When input is a 2-D tensor this is equivalent to transpose(input, 0, 1).

In [20]:

x = torch.randn(()) # here, we create a 0-dimension tensor
print(x)
torch.t(x) # get the transpose of the tensor

tensor(0.5451)

Out[20]:

tensor(0.5451)

When you get the transpose of a 0-dimension tensor, you obtain the same tensor.

In [21]:

x = torch.randn((3)) # here, we create a 1-dimension tensor
print(x)
torch.t(x)

tensor([-0.3837, -0.9967, -0.2112])

Out[21]:

tensor([-0.3837, -0.9967, -0.2112])

When you get the transpose of a 1-dimension tensor, you obtain the same tensor.

In [22]:

x = torch.randn(2, 3) # We create a tensor with 2 rows and 3 columns, and fill it with random values
print(x)
torch.t(x)

tensor([[ 0.9592,  0.4125,  1.0104],
        [-0.3084,  1.5123, -2.2628]])

Out[22]:

tensor([[ 0.9592, -0.3084],
        [ 0.4125,  1.5123],
        [ 1.0104, -2.2628]])

When you get the transpose of a 2-dimensions tensor, you obtain a switched version of its rows with its columns.

In [23]:

x = torch.randn(2, 3, 4) # 3-dimensions tensor
print(x)
torch.t(x)

tensor([[[ 0.5688,  0.4226, -0.6718, -0.0961],
         [ 1.1667,  0.7585, -2.1319,  0.2441],
         [ 0.4130, -1.1373, -1.2946,  0.4909]],

        [[-0.7102,  0.2270,  1.4272, -0.4438],
         [ 1.4970, -0.5961, -0.6305,  0.4257],
         [-1.4224, -0.3125,  0.0097,  1.2551]]])

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-23-90e71df1462f> in <module>()
      1 x = torch.randn(2, 3, 4) # 3-dimensions tensor
      2 print(x)
----> 3 torch.t(x)

RuntimeError: t() expects a tensor with <= 2 dimensions, but self is 3D

In this example, we can see that if you transpose a tensor with 3 or more dimmensions, you will get an error.

We can get the transpose of a tensor when we want to reveal some properties of the linear transformation, such as symmetry.

torch.numel(input)

Returns the total number of elements in the input tensor.

In [24]:

a = torch.randn(1, 2, 3, 4, 5) # 5-dimensions tensor
print(a)
torch.numel(a)

tensor([[[[[ 0.5683,  0.3660,  0.1169,  0.6211,  0.1339],
           [ 0.7302,  0.4227,  0.5289,  0.0528, -1.2226],
           [-1.5867,  0.2716, -1.9211, -0.2082,  1.2038],
           [ 1.2828,  0.8668, -0.8027, -0.6341,  0.1237]],

          [[-0.7317, -1.2314, -0.3423,  1.9565, -0.9173],
           [ 1.1375,  0.9102,  0.1213,  0.6792,  1.3508],
           [ 0.5755, -0.1915, -1.0419, -2.0169,  1.9030],
           [-0.8302,  0.0087,  1.0707,  1.6864,  0.7094]],

          [[ 0.8155, -0.3658,  0.5485, -1.0085, -0.0369],
           [ 1.1497,  0.8193, -2.2531,  0.2470, -0.9620],
           [ 0.9751, -1.6460,  0.5189, -1.3889, -0.8977],
           [-0.0137,  0.7719,  0.0343,  0.6531, -1.4094]]],


         [[[-2.1804, -1.0201,  0.0127, -2.3253, -0.2246],
           [-0.4352,  1.8872, -0.3121,  0.1729,  0.5579],
           [ 1.1935, -0.7444, -1.4449,  1.0257,  0.4009],
           [-1.4843, -0.2777,  0.7592,  0.2888,  1.3752]],

          [[-0.4171,  0.6554,  1.5389, -0.6104, -0.9522],
           [-0.1757,  0.7910, -0.2597, -2.4339,  0.1102],
           [-0.5576,  0.0128, -0.2287,  1.6276,  1.6970],
           [ 1.3917,  0.1737, -0.1108,  0.5641, -0.8908]],

          [[ 0.1575,  0.2302,  0.3282, -0.3098, -1.2024],
           [-1.3616, -0.7325,  0.0336, -1.7040, -0.1361],
           [ 0.7067,  0.2282, -0.5501,  0.3377,  1.1849],
           [-1.0774,  1.1304, -0.2066,  0.2518,  0.1125]]]]])

Out[24]:

120

In this example, we created a 5-dimensions tensor, and obtained the number of elements.

In [25]:

a = torch.zeros(4,4) # 2-dimensions tensor
print(a)
torch.numel(a)

tensor([[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]])

Out[25]:

16

Here, we created a 2-dimensions tensor, and obtained the number of elements.

In [26]:

a = [2,3]
print(a)
torch.numel((2,3))

[2, 3]

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-26-403f38c975eb> in <module>()
      1 a = [2,3]
      2 print(a)
----> 3 torch.numel((2,3))

TypeError: numel(): argument 'input' (position 1) must be Tensor, not tuple

Make sure to pass a tensor as the parameter of the numel function. Otherwise, you will get an error like this.

We can use the numel function when we want to know the number of elements we are handling in a tensor.

Conclusion

As you can see from this brief article, PyTorch has many features that helps you to handle tensors. There are many more functionalities that you can review from the PyTorch official documentation.