The sync Package Unveiled - WaitGroup: Orchestrating Concurrent Goroutine Completion
James Reed
Infrastructure Engineer · Leapcell

Go's concurrency model, built around goroutines and channels, is incredibly powerful and elegant. However, with power comes responsibility – managing these concurrent processes effectively is crucial for building robust and reliable applications. One of the fundamental tools in your concurrency toolkit, provided by the sync
package, is sync.WaitGroup
.
The sync.WaitGroup
type is designed to wait for a collection of goroutines to finish. It acts as a counter that can be incremented and decremented. When the counter reaches zero, the Wait
method unblocks. This simple mechanism is profoundly useful for scenarios where you launch multiple goroutines and need to ensure they've all completed their work before the main goroutine (or any other goroutine) proceeds.
Why WaitGroup
? The Problem It Solves
Imagine a scenario where your application needs to process a large number of tasks concurrently. You decide to launch a goroutine for each task. Without a mechanism to wait for these goroutines, your main program might exit prematurely, or attempt to use results that haven't been computed yet.
Consider this naive, problematic example:
package main import ( "fmt" "time" ) func processTask(id int) { fmt.Printf("Task %d started\n", id) time.Sleep(time.Duration(id) * 100 * time.Millisecond) // Simulate work fmt.Printf("Task %d finished\n", id) } func main() { for i := 1; i <= 5; i++ { go processTask(i) } fmt.Println("All tasks launched. Exiting main.") // What happens here? Many tasks might not complete before main exits. }
If you run the above code, you'll likely observe that not all "Task X finished" messages appear, or they appear in an uncoordinated manner after "Exiting main." The main
goroutine doesn't wait for the processTask
goroutines to complete. This is precisely the problem sync.WaitGroup
addresses.
How sync.WaitGroup
Works
sync.WaitGroup
exposes three key methods:
Add(delta int)
: Increments theWaitGroup
counter bydelta
. You typically call this before launching a new goroutine to indicate that another goroutine is joining the group. Ifdelta
is negative, the counter is decremented.Done()
: Decrements theWaitGroup
counter by one. This is typically called at the very end of a goroutine's execution (often usingdefer
) to signal its completion. It's equivalent toAdd(-1)
.Wait()
: Blocks the calling goroutine until theWaitGroup
counter becomes zero. This means all the goroutines that wereAdd
ed have also calledDone
.
Implementing WaitGroup
Correctly
Let's refactor our previous example using sync.WaitGroup
:
package main import ( "fmt" "sync" "time" ) func processTaskWithWG(id int, wg *sync.WaitGroup) { // Crucially, call Done() before exiting the goroutine. // defer ensures it's called even if an error occurs. defer wg.Done() fmt.Printf("Task %d started\n", id) time.Sleep(time.Duration(id) * 100 * time.Millisecond) // Simulate work fmt.Printf("Task %d finished\n", id) } func main() { var wg sync.WaitGroup // Declare a WaitGroup for i := 1; i <= 5; i++ { wg.Add(1) // Increment the WaitGroup counter for each new goroutine go processTaskWithWG(i, &wg) // Pass the WaitGroup by pointer } // Wait for all goroutines to complete wg.Wait() fmt.Println("All tasks complete. Exiting main.") }
When you run this revised code, you will consistently see all "Task X finished" messages before "All tasks complete. Exiting main." The main
goroutine now correctly waits for all the processTaskWithWG
goroutines to finish their execution.
Important Considerations:
- Pointers vs. Values: Always pass
WaitGroup
by pointer (*sync.WaitGroup
) to goroutines. If you pass it by value, each goroutine receives a copy of theWaitGroup
, and itsDone()
call will only decrement its local copy, not the originalWaitGroup
in themain
goroutine. This is a common pitfall. Add
Beforego
: Callwg.Add(1)
before starting the new goroutine. If you callAdd
inside the goroutine, there's a race condition where themain
goroutine might executewg.Wait()
before the new goroutine has a chance to increment the counter, leading toWait()
unblocking prematurely.defer wg.Done()
: Usingdefer
to callwg.Done()
ensures that the counter is decremented even if the goroutine panics or returns early due to an error. This prevents theWait
method from blocking indefinitely (a deadlock).
A More Complex Example: Fan-Out and Fan-In Pattern
WaitGroup
is excellent for implementing the fan-out/fan-in pattern, where you distribute work to multiple workers (fan-out) and then collect their results (fan-in). While WaitGroup
itself doesn't collect results (channels are typically used for that), it ensures all workers have finished before you proceed to process the collected results.
Let's imagine fetching data from multiple URLs concurrently and then processing all responses.
package main import ( "fmt" "io/ioutil" "net/http" "sync" "time" ) // fetchData fetches data from a URL and sends it to a channel func fetchData(url string, results chan<- string, wg *sync.WaitGroup) { defer wg.Done() // Ensure WaitGroup counter is decremented fmt.Printf("Fetching %s...\n", url) resp, err := http.Get(url) if err != nil { results <- fmt.Sprintf("Error fetching %s: %v", url, err) return } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { results <- fmt.Sprintf("Error reading body from %s: %v", url, err) return } results <- fmt.Sprintf("Content from %s (first 50 chars): %s", url, string(body)[:min(50, len(body))]) } func min(a, b int) int { if a < b { return a } return b } func main() { urls := []string{ "http://example.com", "http://google.com", "http://bing.com", "http://invalid-url-example.com", // This will cause an error } var wg sync.WaitGroup // Create a buffered channel to hold results. Size it to the number of URLs. // This prevents the sender from blocking if the receiver is slow. results := make(chan string, len(urls)) fmt.Println("Starting data fetching...") for _, url := range urls { wg.Add(1) // Always add before launching the goroutine go fetchData(url, results, &wg) } // Start a goroutine to close the results channel after all fetching goroutines are done. // This is important so the range loop over "results" channel knows when to terminate. go func() { wg.Wait() // Wait for all fetchData goroutines to complete close(results) // Close the channel }() // Process results as they come in, or after all are collected. // Using a range loop on a channel ensures we process all results. fmt.Println("\nProcessing fetched data:") for res := range results { fmt.Println(res) } fmt.Println("\nAll data processing complete. Exiting main.") time.Sleep(time.Second) // Give a moment to ensure output order }
In this fan-out/fan-in
example:
- We launch a
fetchData
goroutine for each URL, usingwg.Add(1)
to track each one. - Each
fetchData
goroutine callswg.Done()
upon completion (or error). - A separate anonymous goroutine is responsible for calling
wg.Wait()
and thenclose(results)
. Closing the channel signals to thefor res := range results
loop that no more values will be sent, allowing it to terminate gracefully. Without this, the main goroutine'srange results
loop would block indefinitely after all items were processed. - The
main
goroutine then iterates over theresults
channel, printing each fetched piece of data.
This pattern is very common and powerful for concurrent data processing.
Best Practices and Common Pitfalls
- Don't
Add
inside the goroutine: As discussed, callingwg.Add(1)
inside the spawned goroutine can lead to race conditions. Always increment the counter before spawning the goroutine. - Always
defer wg.Done()
: It's the most robust way to ensure the counter is decremented. - Pass by pointer:
sync.WaitGroup
must be passed by pointer (*sync.WaitGroup
) to goroutines. - Avoid
Add
ing afterWait
has been called: OnceWait()
returns, theWaitGroup
can theoretically be reused. However,Add
ing to the counter afterWait()
has been called can lead to undefined behavior or panics if another goroutine is still or newly waiting on the sameWaitGroup
instance. It's generally safer to create a newWaitGroup
for each batch of concurrent tasks, especially ifWait()
is called in a loop. - Zero value
WaitGroup
: AWaitGroup
can be used directly after declaration (its zero value is ready to use), there's no need to initialize it withsync.WaitGroup{}
.
Conclusion
sync.WaitGroup
is an indispensable tool in Go's concurrency toolbox. It provides a simple yet effective mechanism for coordinating the completion of multiple goroutines, preventing common concurrency bugs like premature exits and race conditions. By mastering its Add
, Done
, and Wait
methods, and adhering to best practices, you can build more robust, predictable, and performant concurrent applications in Go. It forms the backbone for many more complex concurrent patterns and is a fundamental building block for any serious Go developer.