李宏毅机器学习课程（1）Pytorch Tutorial

Pytorch作为深度学习常用模块库，里面包含了很多深度学习相关的操作和模块，底层由C++实现，使用张量进行计算，可使用GPU加速。

Pytorch Documentation

参考网址：https://pytorch.org/docs/stable/

常用的模块有：

• torch.nn -> neural nework
• torch.optim -> optimization algorithms
• torch.utils.data -> dataset, dataloader

Pytorch Documentation Example

Function inputs and outputs

Data type and explanation of each input

Some functions behave differently with different inputs

在PyTorch中，参数(parameters)和关键字参数(keyword arguments)是两种不同的概念。

参数是模型中的可学习参数，例如线性层中的权重。它们通常被定义为模型的nn.Module的成员变量，并且可以通过调用nn.Module的成员函数parameters()返回。

关键字参数是函数的额外参数，它们是可选的。它们可以通过向函数提供以参数名称为关键字的额外参数的形式来提供，例如：nn.Linear(in_features, out_features, bias=True)。

三种不同输入对应的torch.max

Common Errors

Tensor on Different Device to Model

model = torch.nn.Linear(5,1).to("cuda:0") 
x = torch.randn(5).to("cpu") 
y = model(x) 

Tensor for * is on CPU, but expected them to be on GPU

=> send the tensor to GPU

x = torch.randn(5).to("cuda:0") 
y = model(x) 
print(y.shape)

Mismatched Dimensions

x = torch.randn(4,5)
y = torch.randn(5,4)
z = x + y

The size of tensor a (5) must match the size of tensor b (4) at non-singleton dimension 1

=> the shape of a tensor is incorrect, use transpose, squeeze, unsqueeze to align the dimensions

y = y.transpose(0,1)
z = x + y
print(z.shape)

Cuda Out of Memory

import torch
import torchvision.models as models
resnet18 = models.resnet18().to( "cuda:0") # Neural Networks for Image Recognition
data = torch.randn( 512,3,244,244) # Create fake data (512 images)
out = resnet18(data.to( "cuda:0")) # Use Data as Input and Feed to Model
print(out.shape)

CUDA out of memory. Tried to allocate 350.00 MiB (GPU 0; 14.76 GiB total capacity; 11.94 GiB already allocated; 123.75 MiB free; 13.71 GiB reserved in total by PyTorch)

=> The batch size of data is too large to fit in the GPU. Reduce the batch size.

Mismatched Tensor Type

import torch.nn as nn
L = nn.CrossEntropyLoss()
outs = torch.randn(5,5)
labels = torch.Tensor([1,2,3,4,0])
lossval = L(outs,labels) # Calculate CrossEntropyLoss between outs and labels

expected scalar type Long but found Float

=> labels must be long tensors, cast it to type “Long” to fix this issue

labels = labels.long()
lossval = L(outs,labels)
print(lossval)