The Wait-Job command waits for all of the jobs in the input pipeline to complete. Wait-Job supports a degree of filtering and offers a timeout to define jobs to wait for.

In some cases, it is desirable to pull off output from jobs as they complete. This can be solved by creating a while or do loop in PowerShell, reacting to jobs as the state changes:

while (Get-Job -State Running) {
    $jobs = Get-Job -State Completed
    $jobs | Receive-Job
    $jobs | Remove-Job
    Start-Sleep -Seconds 1
}

A while loop does not have an output pipeline, if output is to be piped to another command it would need to be piped within the loop. For example, if the job output were filling a CSV file, Export-Csv would be added inside the loop and the Append parameter would be used:

while (Get-Job -State Running) {
    $jobs = Get-Job -State Completed
    $jobs | Receive-Job | Export-Csv output.csv -Append
    $jobs | Remove-Job
    Start-Sleep -Seconds 1
}

This technique is useful if the job is returning a large amount of data. Streaming output to a file as jobs complete will potentially help manage memory usage across a larger number of jobs.

This approach can be combined with the snippet, which limits the number of concurrent jobs. The tweak is shown as follows:

$listOfJobs = 1..50
$jobs = foreach ($job in $listOfJobs) {
    while (@(Get-Job -State Running).Count -gt 10) {
        Start-Sleep -Seconds 10
    }

    Start-Job { Start-Sleep -Seconds (Get-Random -Minimum 10 -Maximum 121) }
    Get-Job -State Completed | Receive-Job | Export-Csv output.csv -Append
}

$jobs | Wait-Job | Receive-Job | Export-Csv output.csv -Append

The final line is required to wait for and then receive the jobs that were still running when the last job was started.