stride

The stride dictates the number of positions the inputs shift across batches, creating a sliding window approach. It is used for example in the DataLoader; in the following example we see that a stride = 1 shifts the second batch by one position compared to the first one, with the target portion as an anchor point for the shift.

dataloader = create_dataloader_v1(raw_text, batch_size=1, 
                                  max_length=4, #usually much larger (e.g. 256)
                                  stride=1, 
                                  shuffle=False)

data_iter = iter(dataloader)
inputs, targets = next(data_iter)
print("Inputs:\n", inputs)
print("\nTargets:\n", targets)

```
Inputs:
 tensor([[  40,  367, 2885, 1464]])

Targets:
 tensor([[ 367, 2885, 1464, 1807]])
```

#with a stride=1 the second batch will be shifted by 1 position 
inputs, targets = next(data_iter)
print("Inputs:\n", inputs)
print("\nTargets:\n", targets)

```
Inputs:
 tensor([[ 367, 2885, 1464, 1807]])

Targets:
 tensor([[2885, 1464, 1807, 3619]])
```

The anchor point concept is mote visible in an example where we use batch_size = 2 , and let’s use a larger stride = 2 too:

dataloader = create_dataloader_v1(raw_text, batch_size=2, 
                                  max_length=4,
                                  stride=2, 
                                  shuffle=False)

data_iter = iter(dataloader)
inputs, targets = next(data_iter)
print("Inputs:\n", inputs)
print("\nTargets:\n", targets)

```
Inputs:
 tensor([[  40,  367, 2885, 1464],
        [2885, 1464, 1807, 3619]])

Targets:
 tensor([[ 367, 2885, 1464, 1807],
        [1464, 1807, 3619,  402]])
```

#with a stride=2 the second batch will be shifted by 2 position 
#from the last target sequence
inputs, targets = next(data_iter)
print("Inputs:\n", inputs)
print("\nTargets:\n", targets)

```
Inputs:
 tensor([[ 1807,  3619,   402,   271],
        [  402,   271, 10899,  2138]])

Targets:
 tensor([[ 3619,   402,   271, 10899],
        [  271, 10899,  2138,   257]])
```