Go Concurrency Learning: The Necessity of a Channel from Factorial Cases

In Go language concurrent programming, goroutine is the core of efficient concurrency, but the communication and synchronization between multiple goroutines is often a place for beginners to easily step on pits. This article will take ‘calculate the factorials of 1-200 and store them into Map’ as a case, and combine the problems encountered in the learning process, such as no locks and no sleep, etc. Gradually analyze the limitations of global variable lock synchronization, and finally understand the necessity of channel, and record your own learning experience and thinking.

1. Case requirements and initial ideas
The core requirements of this learning: use goroutine to calculate the factorial of each number 1-200, put the results into the map, and finally print all the results.
The initial idea is very direct, refer to the boot in PPT:

1. Write a factorial calculation function, which is responsible for calculating the factorial of a single number and storing the result into the map;
2. Start multiple goroutines (corresponding to each number of 1-200), and perform factorial calculation and result storage in concurrently;
3. Set the map as a global variable, which is convenient for all goroutines to access and modify.
   According to this idea, I first wrote the basic code framework, but in the actual operation, there are various problems, and it is these problems that allow me to gradually understand the core logic of Go concurrent synchronization.

2. Problem dismantling: three kinds of abnormal scene analysis
Combined with the content of the PPT and my own debugging process, I have divided the problems into three categories: ‘no lock scene’, ‘no sleeping scene’, ‘locking and hibernating still have hidden danger scenarios’, and analyze the essence and phenomenon of the problem one by one.

Scenario 1: Lock-free state – concurrent map writes
First, I removed the mutex (lock.lock() and lock.unlock()) in the code, leaving only the global map and goroutine, the code core fragment is as follows:

var myMap = make(map[int]int, 10)

// 无锁版本test函数
func test(n int) {
	res := 1
	for i := 1; i <= n; i++ {
		res *= i
	}
	myMap[n] = res // 直接写入全局map，无任何锁保护
}

func main() {
	for i := 1; i <= 20; i++ {
		go test(i) // 启动多个Goroutine
	}
	// 未加休眠，未加锁打印
	for i, v := range myMap {
		fmt.Printf("map[%d]=%d\n", i, v)
	}
}

Run the above code directly, the error is not concurrent map writes, but fatal error: concurrent map iteration and map write, the actual operation error is as follows:

/app/go-atguigu/goroutine-demo02 # go run main.go
map[1]=1
map[2]=2
map[8]=40320
fatal error: concurrent map iteration and map write

goroutine 1 [running]:
internal/runtime/maps.fatal({0x4ccdcf?, 0x38ce22c48040?})
        /usr/local/go/src/runtime/panic.go:1181 +0x18
internal/runtime/maps.(*Iter).Next(0x0?)
        /usr/local/go/src/internal/runtime/maps/table.go:819 +0x86
main.main()
        /app/go-atguigu/goroutine-demo02/main.go:33 +0x119
exit status 2

Only add hibernation code to the main thread (//time.sleep(time.second * 5)), let the main thread wait for all goroutines to perform write operations first, and avoid ‘read + write’ concurrent, and the error will become concurrent Map writes, the specific error is as follows:

/app/go-atguigu/goroutine-demo02 # go run main.go
fatal error: concurrent map writes

goroutine 19 [running]:
internal/runtime/maps.fatal({0x4c8de5?, 0x0?})
        /usr/local/go/src/runtime/panic.go:1181 +0x18
main.test(...)
        /app/go-atguigu/goroutine-demo02/main.go:25
created by main.main in goroutine 1
        /app/go-atguigu/goroutine-demo02/main.go:30 +0x4b
exit status 2

Here we need to focus on correcting a detail: when the above-mentioned unlocked code runs directly, the error is not the concurrent map writes mentioned in the ppt, but the concurrent map iteration and map write – The reason is that the main thread performs the read operation of map (loop printing) without adding hibernation, and there are multiple goroutines to perform the write operation of the map, which belongs to the ‘read + write’ concurrent conflict; Only add hibernation code to the main thread, let the main thread pause the read operation, and wait for all the goroutines to execute the write operation first. Map writes error, these two errors are essentially the concurrency security problem of MAP, and it is also the ‘resource competition problem’ that is emphasized in the PPT. as shown in Figure 1

The essence of the problem: Map in GO is a non-concurrency-safe data structure (this is also the basic knowledge point of Go concurrent programming), and multiple goroutines are at the same time for M When AP performs write operations, there will be resource competition – multiple goroutines will also seize the memory resources of the map at the same time, resulting in confusion of data writing and program crashing. As PPT said, the method of judging whether there is resource competition is very simple, adding the -race parameter when compiling the program, you can clearly see the competition point.
Supplementary note: Not only will there be competition for write operations, but even if it is ‘reading + writing’ at the same time, there will also be resource competition. This is because global variables are shared by all goroutines. Without any restrictions, the read and write operations of multiple goroutines will interfere with each other, which is also an inherent problem of ‘shared memory’ mode in concurrent programming.

Scenario 2: No sleep state – the main thread exits early, the result is lost
After solving the lock-free problem, I added a mutex to ensure the read and write safety of the map, but the sleep code of the main thread was removed ( //time.sleep(time.second * 5)), the code core fragment is as follows:

package main
import (
	"fmt"
	//"time"
	"sync"
)

// 需求：现在要计算 1-200 的各个数的阶乘，并且把各个数的阶乘放入到map中。
// 最后显示出来。要求使用goroutine完成 

// 思路
// 1. 编写一个函数，来计算各个数的阶乘，并放入到 map中.
// 2. 我们启动的协程多个，统计的将结果放入到 map中
// 3. map 应该做出一个全局的.

var (
	myMap = make(map[int]int, 10)  
	//声明一个全局的互斥锁
	//lock 是一个全局的互斥锁， 
	//sync 是包: synchornized 同步
	//Mutex : 是互斥
	lock sync.Mutex
)

// test 函数就是计算 n!, 让将这个结果放入到 myMap
func test(n int) {
	
	res := 1
	for i := 1; i <= n; i++ {
		res *= i
	}

	//这里我们将 res 放入到myMap
	//加锁
	lock.Lock()
	myMap[n] = res //concurrent map writes?
	//解锁
	lock.Unlock()
}

func main() {

	// 我们这里开启多个协程完成这个任务[200个]
	for i := 1; i <= 20; i++ {
		go test(i)
	}


	//休眠10秒钟【第二个问题 】
	//time.Sleep(time.Second * 5)

	//这里我们输出结果,变量这个结果
	lock.Lock()
	for i, v := range myMap {
		fmt.Printf("map[%d]=%d\n", i, v)
	}
	lock.Unlock()

}

After running, the console either prints empty results, or only prints a small number of results, and most of the factorial results are lost. as shown in Figure 2

/app/go-atguigu/goroutine-demo02 # go run main.go
/app/go-atguigu/goroutine-demo02 # go run main.go
map[1]=1
map[3]=6
map[7]=5040
map[2]=2
/app/go-atguigu/goroutine-demo02 # go run main.go
/app/go-atguigu/goroutine-demo02 # go run main.go
map[10]=3628800
map[15]=1307674368000
map[2]=2
map[13]=6227020800
map[14]=87178291200
map[6]=720
map[12]=479001600
map[5]=120
map[11]=39916800
map[1]=1
map[3]=6
map[4]=24
map[8]=40320
map[7]=5040
map[9]=362880
/app/go-atguigu/goroutine-demo02 # 

The essence of the problem: goroutine is a lightweight execution unit, by go Runtime scheduling, and the main thread (the goroutine corresponding to the main function) will not wait for other goroutines to finish executing, and will continue to execute the subsequent code, and exit the program after executing the print operation. Once the main thread exits, all unexecuted goroutines will be forced to be destroyed, causing their calculation results to not be written to the map, and the final result will be lost.
This is the ‘second problem’ mentioned in PPT: the execution rhythm of the main thread and goroutine is inconsistent, and the main thread does not wait for other goroutines to complete the task, so that the result cannot be obtained normally.

Scenario 3: Lock and sleep – it seems feasible, but there are still hidden dangers and inflexibility
Combined with the first two problems, I added the mutex lock and the main thread sleep code at the same time, that is, the ‘final code’ provided by the user, which seems to solve all the problems, but after in-depth debugging, I found that this method still has hidden dangers and is not flexible enough.
First of all, on ‘Why do you need to lock it when printing’, the key explanation is given in PPT: although we subjectively estimate that 10 seconds is enough for all goroutines to be executed, the main thread does not know the execution status of other goroutines. Even if the sleeping time is set, the underlying scheduling may still have the situation that ‘the main thread will start printing before the main thread starts to print’. At this time, the write operation (read map) and the write operation of goroutine will generate resource competition, so lock protection is also required when printing.
Second, the core hidden danger of this method is that the sleep time cannot be accurately controlled.

• If the sleep time is set for too long (such as 10 seconds), it will cause unnecessary waiting, waste system resources, and reduce program efficiency;
• If the sleep time setting is too short (such as 1 second), some goroutines may not have been executed, and there will still be a problem of losing the result;
• In actual development, the execution time of goroutine is affected by factors such as task complexity and system load, and cannot be accurately estimated in advance. Hard-coded sleep time will lead to extremely poor program stability and scalability.
  In addition, there is a core problem in the way of global variable lock synchronization: multiple goroutines communicate with each other, all read and write operations are It needs to be controlled by locks, which not only increases the code complexity, but also prone to deadlocks (such as forgetting to unlock), unreasonable lock granularity, etc., which is not conducive to subsequent maintenance and expansion. As PPT said, this method is ‘imperfect’ and cannot meet the needs of flexible and safe concurrent communication.

3. Summary: Why is the channel necessary?
Through the analysis of the above three scenarios, combined with the guidance in PPT, I deeply understand the necessity of the emergence of Channel – it is to solve the many drawbacks of ‘global variable lock synchronization’, and realize more secure and flexible communication and synchronization between goroutines.
The necessity of the channel is summarized as follows:

1. Solve the problem of resource competition without manual locking
   Channel is the ‘pipe’ for communication between goroutines in the Go language, following the design philosophy of ‘sharing memory through communication, rather than communicating through shared memory’. It is concurrent and safe. When multiple goroutines send and receive data through the channel, they can avoid the problem of resource competition without manually adding mutexes.
   In the factorial case, if the channel is used instead of the global map, the goroutine calculates the factorial, and the result is sent to the chan In NEL, the main thread receives the results from the channel and prints it, which can completely avoid the concurrent read and write problems of maps, and the code is more concise and safer.
2. Precise sync goroutine, no manual sleep required
   Channel has ‘blocking characteristics’: the send and receive operations of the unbuffered channel will block each other until both parties are ready; there is a buffer channel that blocks the send when the buffer is full, and the buffer will be blocked when the buffer is empty. Using this feature, the main thread can accurately wait for all the goroutines to be executed without manually setting the sleep time.
   For example, we can start 200 goroutines, and after each goroutine calculates the factorial, send a ‘complete signal’ to the channel, After the main thread receives 200 signals, it performs a print operation, which can ensure that all goroutines have been executed, which will neither waste time nor lose results.
3. Reduce code complexity and improve maintainability
   The method of global variable lock synchronization requires manual management of locking and unlocking of the lock, which is prone to problems such as omission, unlocking, and deadlock, and the sharing of global variables will lead to an increase in code coupling. Channel integrates ‘data transmission’ and ‘synchronous control’. There is no need to share global resources between goroutines. Through channel, data and signals are transmitted, and the code logic is clearer and the maintenance cost is lower.
4. Adapt to more complex concurrency scenarios
   With the complexity of concurrency scenarios (such as multi-producer, multi-consumer mode), the method of global variable locking will become more and more difficult to control, and the channel supports one-way channel, select multiplexing and other features, which can easily cope with complex concurrent communication needs. For example, we can avoid the situation of one party waiting for the other party by balancing the speed of the producer (the goroutine of the factorial) and the consumer (the main thread of the print result) by having a buffer channel.

4. Learning perception
Through the learning of this factorial case, I deeply realized that goroutine makes Go’s concurrency simple, but the real difficulty lies in the communication and synchronization between goroutines. Global variable lock synchronization is a basic solution, but its limitations are obvious, and channel is the core primitive of Go concurrent programming, which perfectly solves these problems.
The design philosophy of the Go language tells us: ‘Don’t communicate through shared memory, but share memory through communication’. Channel is the embodiment of this philosophy, which makes concurrent programming safer, more elegant and maintainable. This learning not only solves the specific problems in the factorial case, but also allows me to understand the core logic of Go concurrency, which has laid the foundation for subsequent learning of more complex concurrent scenarios (such as worker pool and select multiplexing).
In the follow-up, I will continue to learn the use of Channel in depth (no buffering, buffering, one-way channels, etc.), combined with more cases, and skillfully use the channel to realize the communication and synchronization between Goroutine, and truly grasp the essence of Go concurrent programming.