Go: how to create a download progress indicator for an HTTP request

To implement a command line utility in Go that downloads a file from a URL and shows the download progress in the console, you can follow the steps outlined in this article.

Here are the required steps:

1. Set up the basic command line utility: Use the flag package to parse command line arguments.

2. Handle the HTTP request: Use the http package to send a request to the provided URL and receive the file's content.

3. Display download progress: Track the amount of data read from the response and update the console with the progress.

We can write the following code:

package main

import (
    "flag"
    "fmt"
    "io"
    "net/http"
    "os"
    "strings"
)

func main() {
    // Parse URL from command line argument
    flag.Parse()
    args := flag.Args()
    if len(args) < 1 {
        fmt.Println("Please provide a URL")
        return
    }
    url := args[0]

    // Start the download
    err := downloadFile(url)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error downloading file: %v\n", err)
        os.Exit(1)
    }
}

func downloadFile(url string) error {
    resp, err := http.Get(url)
    if err != nil {
        return err
    }
    defer resp.Body.Close()

    // Create the file
    segments := strings.Split(url, "/")
    fileName := segments[len(segments)-1]
    file, err := os.Create(fileName)
    if err != nil {
        return err
    }
    defer file.Close()

    // Progress reporting
    totalBytes := resp.ContentLength
    var downloadedBytes int64 = 0

    // Create a buffer to write with 32 KB chunks
    buffer := make([]byte, 32*1024)
    for {
        n, err := resp.Body.Read(buffer)
        if n > 0 {
            _, writeErr := file.Write(buffer[:n])
            if writeErr != nil {
                return writeErr
            }
            downloadedBytes += int64(n)
            fmt.Printf("\rDownloading... %d%% complete", 100*downloadedBytes/totalBytes)
        }
        if err == io.EOF {
            break
        }
        if err != nil {
            return err
        }
    }

    fmt.Println("\nDownload completed successfully")
    return nil
}

The above code is made up of the following sections:

1. Command Line Arguments Handling:

   • flag.Parse(): Parses the command line arguments provided to the program.
   • flag.Args(): Returns the non-flag command line arguments (i.e., arguments that are not options starting with a dash).
   • If no arguments are provided, the program prints a message asking for a URL and exits.

2. Downloading the File:

   • The URL obtained from the command line arguments is passed to the downloadFile function.
   • If there is an error in downloading, it prints the error message to the standard error stream and exits with a status code of 1.

3. HTTP Request:

   • http.Get(url): Sends an HTTP GET request to the specified URL.
   • If the request fails, the error is returned immediately.
   • defer resp.Body.Close(): Ensures that the response body is closed when the function exits. This prevents resource leaks.

4. File Handling:

   • The URL's last segment (after the last /) is used as the filename.
   • os.Create(fileName): Creates a new file for writing. If the file already exists, it will be truncated.
   • defer file.Close(): Ensures that the file is closed when the function exits.

5. Data Transfer:

   • io.Copy(file, resp.Body): Copies data from the HTTP response body directly to the file. This method is efficient as it streams the data, handling large files without consuming a lot of memory.
   • If an error occurs during copying, it is returned.

6. Completion Message:

   • Prints a message indicating that the download was successful.

The downloadFile function also adds progress reporting. It does this by reading chunks of data from the response and writing them to the file in a loop. After each chunk, it updates a console progress meter. Here are the key additions for progress reporting:

• Progress Variables:

  • totalBytes and downloadedBytes track the total size of the file and how much has been downloaded, respectively.

• Buffered Reading and Writing:

  • Data is read in chunks (32 KB in this example). This allows the utility to update the progress after each chunk.

• Progress Display:

  • A simple console progress meter updates after writing each chunk. It calculates the percentage of the total download completed and displays it.

This approach assumes that the server provides the Content-Length header. If it's not available, the progress percentage cannot be calculated, and the utility would need to adjust how progress is displayed.