Many REST interfaces will return large result sets from searches in pages, a sub-set of the results. The techniques used to retrieve each subsequent page can vary from one API to another.

The GitHub API exposes the link to the next page in the HTTP header. This is consistent with RFC 5988 (https://tools.ietf.org/html/rfc5988#page-6).

In PowerShell Core, it is easy to retrieve and view the header when using Invoke-RestMethod:

$params = @{
    Uri                     = 'https://api.github.com/search/issues'
    Body                    = @{
       q = 'documentation state:closed repo:powershell/powershell'
    }
    ResponseHeadersVariable = 'httpHeader'
}
Invoke-RestMethod @params | Select-Object -ExpandProperty items

Once run, the link field of the header may be inspected via the httpHeader variable:

PS> $httpHeader['link']
 <https://api.github.com/search/issues?q=documentation+state%3Aclosed+repo%3Apowershell%2Fpowershell&page=2>; rel="next",
 <https://api.github.com/search/issues?q=documentation+state%3Aclosed+repo%3Apowershell%2Fpowershell&page=34>; rel="last"

PowerShell Core can also automatically follow this link by using the FollowRelLink parameter. This might be used in conjunction with the MaximumFollowRelLink parameter to ensure a request stays within any rate limiting imposed by the web service. See https://developer.github.com/v3/#rate-limiting for the GitHub API, for example:

$params = @{
    Uri                  = 'https://api.github.com/search/issues'
    Body                 = @{
       q = 'documentation state:closed repo:powershell/powershell'
    }
    FollowRelLink        = $true
    MaximumFollowRelLink = 2
}
Invoke-RestMethod @params | Select-Object -ExpandProperty items

Windows PowerShell, unfortunately, cannot automatically follow this link. Nor does the Invoke-RestMethod command expose the header from the response. When working with complex REST interfaces in Windows PowerShell, it is often necessary to fall back to Invoke-WebRequest or even HttpWebRequest classes.

The example that follows uses Invoke-WebRequest in Windows PowerShell to follow the next link in a similar manner to Invoke-RestMethod in PowerShell Core:

# Used to limit the number of times "next" is followed
$followLimit = 2
# The initial set of parameters, describes the search
$params = @{
    Uri = 'https://api.github.com/search/issues'
    # PowerShell will convert this to JSON
    Body = @{
        q = 'documentation state:closed repo:powershell/powershell'
    }
    ContentType = 'application/json'
}
# Just a counter, works in conjunction with followLimit.
$followed = 0

do {
    # Get the next response
    $response = Invoke-WebRequest @params
    # Convert and leave the results as output
    $response.Content | ConvertFrom-Json | Select-Object -ExpandProperty items

    # Retrive the links from the header and find the next URL
    if ($response.Headers['link'] -match '<([^>]+?)>;s*rel="next"') {
        $next = $matches[1]
    } else {
        $next = $null
    }

    # Parameters which will be used to get the next page (next loop iteration)
    $params = @{
        Uri = $next
    }

    # Increment the followed counter
    $followed++
} until (-not $next -or $followed -ge $followLimit)

Because of the flexible nature of REST, implementations of page linking may vary. For example, links may appear in the body of a response instead of the header. Exploration is a requirement when working around a web API.