<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <author>
    <name>Markity</name>
  </author>
  <generator uri="https://hexo.io/">Hexo</generator>
  <id>https://markity.github.io/</id>
  <link href="https://markity.github.io/" rel="alternate"/>
  <link href="https://markity.github.io/atom.xml" rel="self"/>
  <rights>All rights reserved 2026, Markity</rights>
  <subtitle>Sharing knowledge and technology</subtitle>
  <title>Markity Tech Blog</title>
  <updated>2026-06-02T10:19:00.000Z</updated>
  <entry>
    <author>
      <name>Markity</name>
    </author>
    <content>
      <![CDATA[<p>本文讲解http代理的原理，以及Go如何实现这样一个代理，希望能帮助读者更好地理解平时使用的翻墙工具。</p><span id="more"></span><h1 id="何为墙，何为梯"><a href="#何为墙，何为梯" class="headerlink" title="何为墙，何为梯"></a>何为墙，何为梯</h1><p>由于政策问题，中国大陆网络环境无法访问Google，Youtube等网站。这是因为GFW(也就是Great Firewall of China)的干扰。GFW会使用部署在网络链路上的各种过滤干扰机制来干扰双方通信，包括但不限于丢弃报文，fake假的TCP RST包，返回假的DNS解析结果…也就是说，我们普通人的网络直接访问google.com是访问不通的。</p><p>为了解决这个问题，我们可以考虑使用香港&#x2F;台湾的服务器来中转流量，这种服务器需要搭建在外网环境，满足用户能够访问到服务器且服务器能访问到目标网站的条件。</p><p>例如，我想要访问Google，我告诉香港的服务器，我要访问Google，香港服务器帮我访问后返回给我响应结果，这就是完整的一次http请求代理流程。访问HTTPS时，中转服务器未必能看到明文内容，更准确是转发流量或建立隧道，这个略为复杂，后续实现中会详细讲解。</p><p>实现上述技术的工具或系统就叫做梯子。梯子有很多种技术实现，本文聚焦http代理。</p><h1 id="初探https代理协议：代理http请求"><a href="#初探https代理协议：代理http请求" class="headerlink" title="初探https代理协议：代理http请求"></a>初探https代理协议：代理http请求</h1><p>我们先启动一个Go Tcp Server，然后将系统或者浏览器(这里推荐使用Firefox，可以手动设置代理)的http proxy设置为我们的Server。接下来把浏览器发来的信息打印出来。下面是Go Server代码：</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line"><span class="string">&quot;io&quot;</span></span><br><span class="line"><span class="string">&quot;net&quot;</span></span><br><span class="line"><span class="string">&quot;os&quot;</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> &#123;</span><br><span class="line">listener, err := net.ListenTCP(<span class="string">&quot;tcp4&quot;</span>, &amp;net.TCPAddr&#123;IP: net.IPv4(<span class="number">127</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">1</span>), Port: <span class="number">9999</span>&#125;)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="built_in">panic</span>(err)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> &#123;</span><br><span class="line">tcpConn, err := listener.AcceptTCP()</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">continue</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">go</span> handleConn(tcpConn)</span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">handleConn</span><span class="params">(conn *net.TCPConn)</span></span> &#123;</span><br><span class="line"><span class="built_in">println</span>(<span class="string">&quot;new conn&quot;</span>)</span><br><span class="line"><span class="comment">// keep copy conn to stdout</span></span><br><span class="line">io.Copy(os.Stdout, conn)</span><br><span class="line">conn.Close()</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>使用浏览器访问<code>http://test.com/</code>。输出如下:</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">new conn</span><br><span class="line">GET http://test.com/ HTTP/1.1</span><br><span class="line">Host: test.com</span><br><span class="line">Proxy-Connection: keep-alive</span><br><span class="line">Upgrade-Insecure-Requests: 1</span><br><span class="line">User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36</span><br><span class="line">Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7</span><br><span class="line">Accept-Encoding: gzip, deflate</span><br><span class="line">Accept-Language: zh-CN,zh;q=0.9</span><br></pre></td></tr></table></figure><p>可以看到浏览器先会和我们的代理进程建立tcp连接，然后发送想要访问的请求信息。</p><p>接下来我们改造一下代理进程，让它能够解析来自浏览器的http请求，先重温http协议：</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">请求方法[空格]URL[空格]协议版本[\r][\n]</span><br><span class="line">[header1]:[header_val1][\r][\n]</span><br><span class="line">[header2]:[header_val2][\r][\n]</span><br><span class="line">[\r][\n]</span><br><span class="line">[body数据]</span><br></pre></td></tr></table></figure><p>解析代码如下:</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line"><span class="string">&quot;bufio&quot;</span></span><br><span class="line"><span class="string">&quot;fmt&quot;</span></span><br><span class="line"><span class="string">&quot;io&quot;</span></span><br><span class="line"><span class="string">&quot;net&quot;</span></span><br><span class="line"><span class="string">&quot;net/http&quot;</span></span><br><span class="line"><span class="string">&quot;strings&quot;</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="keyword">var</span> fakeResponse = <span class="string">&quot;HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 20\r\n\r\n&lt;h1&gt;Hello World&lt;/h1&gt;&quot;</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> &#123;</span><br><span class="line">listener, err := net.ListenTCP(<span class="string">&quot;tcp4&quot;</span>, &amp;net.TCPAddr&#123;IP: net.IPv4(<span class="number">127</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">1</span>), Port: <span class="number">9999</span>&#125;)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="built_in">panic</span>(err)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> &#123;</span><br><span class="line">tcpConn, err := listener.AcceptTCP()</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">continue</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">go</span> handleConn(tcpConn)</span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">headerClean</span><span class="params">(header http.Header)</span></span> <span class="type">string</span> &#123;</span><br><span class="line"><span class="keyword">if</span> <span class="built_in">len</span>(header) == <span class="number">0</span> &#123;</span><br><span class="line"><span class="keyword">return</span> <span class="string">&quot;&quot;</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">builder := strings.Builder&#123;&#125;</span><br><span class="line"><span class="keyword">for</span> k, v := <span class="keyword">range</span> header &#123;</span><br><span class="line">builder.WriteString(k + <span class="string">&quot;: &quot;</span> + strings.Join(v, <span class="string">&quot;,&quot;</span>) + <span class="string">&quot;\n&quot;</span>)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">result := builder.String()</span><br><span class="line"><span class="keyword">if</span> result[<span class="built_in">len</span>(result)<span class="number">-1</span>] == <span class="string">&#x27;\n&#x27;</span> &#123;</span><br><span class="line"><span class="keyword">return</span> result[:<span class="built_in">len</span>(result)<span class="number">-1</span>]</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">return</span> result</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">handleConn</span><span class="params">(conn *net.TCPConn)</span></span> &#123;</span><br><span class="line"><span class="built_in">println</span>(<span class="string">&quot;new conn&quot;</span>)</span><br><span class="line"><span class="keyword">defer</span> conn.Close()</span><br><span class="line">connBufReader := bufio.NewReader(conn)</span><br><span class="line"><span class="comment">// ReadRequest only support http 1.x</span></span><br><span class="line">req, err := http.ReadRequest(connBufReader)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">body, err := io.ReadAll(req.Body)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">req.Body.Close()</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line">req.Body.Close()</span><br><span class="line"></span><br><span class="line">fmt.Printf(<span class="string">&quot;proto: %v\n&quot;</span>, req.Proto)</span><br><span class="line">fmt.Printf(<span class="string">&quot;url: %v\n&quot;</span>, req.URL.String())</span><br><span class="line">fmt.Printf(<span class="string">&quot;header:\n%v\n&quot;</span>, headerClean(req.Header))</span><br><span class="line">fmt.Printf(<span class="string">&quot;body length: %v\n&quot;</span>, <span class="built_in">len</span>(body))</span><br><span class="line">fmt.Println(<span class="string">&quot;&quot;</span>)</span><br><span class="line"></span><br><span class="line">_, err = conn.Write([]<span class="type">byte</span>(fakeResponse))</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="built_in">println</span>(err)</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>读者可以自行运行上面代码，并且访问<code>http://test.com/</code>进行测试，并且我们fake了一个假的response，打印出了<code>Hello World</code>。</p><h1 id="http1-1-keep-alive"><a href="#http1-1-keep-alive" class="headerlink" title="http1.1 keep-alive"></a>http1.1 keep-alive</h1><p>想必大家经常看到<code>Connection: keep-alive</code>这个header，上面我们打印的浏览器请求也有<code>Proxy-Connection: keep-alive</code>这个header。这是http1.1新引入的一个特性，也就是复用一条TCP连接。</p><p>传统的http1.0使用的模型是: 建立TCP连接-&gt;Browser向Server发送Req-&gt;Server向Browser回应Resp-&gt;关闭TCP连接。而http1.1认为tcp建连接是一个非常高成本的操作，很慢，所以复用一条长连接是很好的，于是引入了<code>Connection</code>头部向服务端请求维持这个长连接。</p><p>如果服务端接受keep-alive，则http resp也带上<code>Connection: keep-alive</code>，如果不接受直接<code>Connection: close</code>即可。浏览器的行为则是串行地发送请求-&gt;接受响应-&gt;发送第二个请求-&gt;接收第二个响应…</p><p>通常浏览器是会同时请求巨多内容的，即使开启了keep-alive复用，本质也是串行的，一样很慢…为了解决这个问题，浏览器 “同时发很多请求”，靠6～8 条并行 TCP 连接，而不是单连接批量发。</p><p><code>Proxy-Connection: keep-alive</code>代表浏览器想要和代理客户端保持长连接，一般合理的代理客户端应该实现它，我上面的<code>Hello World</code>程序是不合理的，没有keep-alive。</p><p>在http2.0引入了但tcp连接多路复用，浏览器可一次性发几十个请求，不用等任何响应，代理&#x2F;服务器可按处理完的顺序返回响应，不用按请求顺序，二进制帧格式，请求&#x2F;响应交错收发，彻底打破串行。本文就不讨论http2.0了，专注http1.x。</p><h1 id="初探http代理-代理https请求"><a href="#初探http代理-代理https请求" class="headerlink" title="初探http代理: 代理https请求"></a>初探http代理: 代理https请求</h1><p>代理https请求会复杂的多，我们一样先访问<code>https://test.com/</code>看看。输出如下：</p><figure class="highlight apache"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="attribute">new</span> conn</span><br><span class="line"><span class="attribute">CONNECT</span> test.com:<span class="number">443</span> HTTP/<span class="number">1</span>.<span class="number">1</span></span><br><span class="line"><span class="attribute">User</span>-Agent: Mozilla/<span class="number">5</span>.<span class="number">0</span> (X11; Ubuntu; Linux x86_64; rv:<span class="number">151</span>.<span class="number">0</span>) Gecko/<span class="number">20100101</span> Firefox/<span class="number">151</span>.<span class="number">0</span></span><br><span class="line"><span class="attribute">Proxy</span>-Connection: keep-alive</span><br><span class="line"><span class="attribute">Connection</span>: keep-alive</span><br><span class="line"><span class="attribute">Host</span>: test.com:<span class="number">443</span></span><br></pre></td></tr></table></figure><p>可以看到这里是一个神秘的CONNECT请求，它的语义是: 为浏览器和目标网站建立一条tcp隧道。代理只负责双向转发流量，tls是浏览器与目标网站进行协商，所以代理看不到明文，也改不了，非常安全。当浏览器想要访问https网站时就会用这个隧道功能，后续浏览器自行和目标网站协商tls握手，然后进行双向tls通讯，不论是运营商还是代理服务器，都对通信内容本身一无所知。</p><p>下面代码展示了Go如何优雅实现这样的双向转发：</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line"><span class="string">&quot;bufio&quot;</span></span><br><span class="line"><span class="string">&quot;fmt&quot;</span></span><br><span class="line"><span class="string">&quot;io&quot;</span></span><br><span class="line"><span class="string">&quot;net&quot;</span></span><br><span class="line"><span class="string">&quot;net/http&quot;</span></span><br><span class="line"><span class="string">&quot;strings&quot;</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> &#123;</span><br><span class="line">listener, err := net.ListenTCP(<span class="string">&quot;tcp4&quot;</span>, &amp;net.TCPAddr&#123;IP: net.IPv4(<span class="number">127</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">1</span>), Port: <span class="number">9999</span>&#125;)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="built_in">panic</span>(err)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> &#123;</span><br><span class="line">tcpConn, err := listener.AcceptTCP()</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">continue</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">go</span> handleConn(tcpConn)</span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">headerClean</span><span class="params">(header http.Header)</span></span> <span class="type">string</span> &#123;</span><br><span class="line"><span class="keyword">if</span> <span class="built_in">len</span>(header) == <span class="number">0</span> &#123;</span><br><span class="line"><span class="keyword">return</span> <span class="string">&quot;&quot;</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">builder := strings.Builder&#123;&#125;</span><br><span class="line"><span class="keyword">for</span> k, v := <span class="keyword">range</span> header &#123;</span><br><span class="line">builder.WriteString(k + <span class="string">&quot;: &quot;</span> + strings.Join(v, <span class="string">&quot;,&quot;</span>) + <span class="string">&quot;\n&quot;</span>)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">result := builder.String()</span><br><span class="line"><span class="keyword">if</span> result[<span class="built_in">len</span>(result)<span class="number">-1</span>] == <span class="string">&#x27;\n&#x27;</span> &#123;</span><br><span class="line"><span class="keyword">return</span> result[:<span class="built_in">len</span>(result)<span class="number">-1</span>]</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">return</span> result</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">handleConn</span><span class="params">(conn *net.TCPConn)</span></span> &#123;</span><br><span class="line"><span class="built_in">println</span>(<span class="string">&quot;new conn&quot;</span>)</span><br><span class="line"><span class="keyword">defer</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> &#123;</span><br><span class="line">fmt.Println(<span class="string">&quot;close&quot;</span>)</span><br><span class="line">&#125;()</span><br><span class="line"><span class="keyword">defer</span> conn.Close()</span><br><span class="line">connBufReader := bufio.NewReader(conn)</span><br><span class="line"><span class="comment">// ReadRequest only support http 1.x</span></span><br><span class="line">req, err := http.ReadRequest(connBufReader)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">_, err = io.ReadAll(req.Body)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">req.Body.Close()</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line">req.Body.Close()</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> req.Method != <span class="string">&quot;CONNECT&quot;</span> &#123;</span><br><span class="line"><span class="keyword">var</span> fakeResponse = <span class="string">&quot;HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 20\r\n\r\n&lt;h1&gt;Hello World&lt;/h1&gt;&quot;</span></span><br><span class="line">_, err = conn.Write([]<span class="type">byte</span>(fakeResponse))</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">conn.Write([]<span class="type">byte</span>(<span class="string">&quot;HTTP/1.1 200 Connection Established\r\n\r\n&quot;</span>))</span><br><span class="line"><span class="comment">// CONNECT, 建立转发</span></span><br><span class="line">remoteConn, err := net.Dial(<span class="string">&quot;tcp&quot;</span>, req.Host)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"><span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> &#123;</span><br><span class="line">io.Copy(remoteConn, conn)</span><br><span class="line">&#125;()</span><br><span class="line">io.Copy(conn, remoteConn)</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>如果设置了http代理并且访问<code>https://www.baidu.com</code>就直接能访问到网页了。</p><h1 id="http代理存在的问题"><a href="#http代理存在的问题" class="headerlink" title="http代理存在的问题"></a>http代理存在的问题</h1><p>我们当前的http代理架构如下:</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">浏览器</span><br><span class="line">  ↓ 明文 HTTP 代理协议/CONNECT要求建立隧道</span><br><span class="line">代理服务器</span><br><span class="line">  ↓ HTTP请求/TCP隧道</span><br><span class="line">目标网站</span><br></pre></td></tr></table></figure><p>这里有两个问题：</p><ul><li>对于HTTP请求代理，比如<code>GET www.baidu.com</code>，浏览器到代理服务器这边是来回是明文的，也就是运营商完全知道你的意图，GFW能轻松看到你们的请求来回拦截流量。</li><li>对于HTTPS请求，虽然建立了隧道，GFW和代理服务器都没办法知道具体的请求内容，但是浏览器向代理服务器发送<code>CONNECT www.youtubu.com</code>也是一个很明显的信号，GFW也能轻易知道你的意图加以拦截。</li></ul><p>究其根本实际上是浏览器到代理服务器的流量来回<strong>没有加密</strong>！</p><p>为了解决这个问题，引入https代理，所谓https代理，实际上就是浏览器到目的服务器这一层的tcp加上了tls握手，其它地方没任何改变，这里我实现一个https代理：</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line"><span class="string">&quot;bufio&quot;</span></span><br><span class="line"><span class="string">&quot;crypto/tls&quot;</span></span><br><span class="line"><span class="string">&quot;fmt&quot;</span></span><br><span class="line"><span class="string">&quot;io&quot;</span></span><br><span class="line"><span class="string">&quot;log&quot;</span></span><br><span class="line"><span class="string">&quot;net&quot;</span></span><br><span class="line"><span class="string">&quot;net/http&quot;</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="keyword">var</span> cert tls.Certificate</span><br><span class="line"><span class="keyword">var</span> tlsConfig *tls.Config</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> &#123;</span><br><span class="line"><span class="keyword">var</span> err <span class="type">error</span></span><br><span class="line">cert, err = tls.LoadX509KeyPair(<span class="string">&quot;server.crt&quot;</span>, <span class="string">&quot;server.key&quot;</span>)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">log.Fatalf(<span class="string">&quot;load cert failed: %v&quot;</span>, err)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">tlsConfig = &amp;tls.Config&#123;</span><br><span class="line">Certificates: []tls.Certificate&#123;cert&#125;,</span><br><span class="line">MinVersion:   tls.VersionTLS12,</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">listener, err := net.ListenTCP(<span class="string">&quot;tcp4&quot;</span>, &amp;net.TCPAddr&#123;IP: net.IPv4(<span class="number">127</span>, <span class="number">0</span>, <span class="number">0</span>, <span class="number">1</span>), Port: <span class="number">9999</span>&#125;)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="built_in">panic</span>(err)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> &#123;</span><br><span class="line">tcpConn, err := listener.AcceptTCP()</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">fmt.Println(err)</span><br><span class="line"><span class="keyword">continue</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="built_in">println</span>(<span class="string">&quot;new conn&quot;</span>)</span><br><span class="line"><span class="keyword">go</span> handleConn(tcpConn)</span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">handleConn</span><span class="params">(conn *net.TCPConn)</span></span> &#123;</span><br><span class="line"><span class="keyword">defer</span> conn.Close()</span><br><span class="line">tlsConn := tls.Server(conn, tlsConfig)</span><br><span class="line"><span class="keyword">defer</span> tlsConn.Close()</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> err := tlsConn.Handshake(); err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="built_in">println</span>(err.Error())</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">fmt.Println(<span class="string">&quot;tls success&quot;</span>)</span><br><span class="line"></span><br><span class="line">connBufReader := bufio.NewReader(tlsConn)</span><br><span class="line"><span class="comment">// ReadRequest only support http 1.x</span></span><br><span class="line">req, err := http.ReadRequest(connBufReader)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> req.Method != http.MethodConnect &#123;</span><br><span class="line">_, err := io.Copy(io.Discard, req.Body)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">req.Body.Close()</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line">req.Body.Close()</span><br><span class="line"><span class="keyword">var</span> fakeResponse = <span class="string">&quot;HTTP/1.1 200 OK\r\nConnection: close\r\nContent-Length: 20\r\n\r\n&lt;h1&gt;Hello World&lt;/h1&gt;&quot;</span></span><br><span class="line">_, err = tlsConn.Write([]<span class="type">byte</span>(fakeResponse))</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> _, err := io.Copy(io.Discard, req.Body); err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line">req.Body.Close()</span><br><span class="line"></span><br><span class="line"><span class="comment">// CONNECT, 建立转发</span></span><br><span class="line">remoteConn, err := net.Dial(<span class="string">&quot;tcp&quot;</span>, req.Host)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">_, _ = tlsConn.Write([]<span class="type">byte</span>(<span class="string">&quot;HTTP/1.1 502 Bad Gateway\r\nConnection: close\r\nContent-Length: 0\r\n\r\n&quot;</span>))</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"><span class="keyword">defer</span> remoteConn.Close()</span><br><span class="line"></span><br><span class="line">_, err = tlsConn.Write([]<span class="type">byte</span>(<span class="string">&quot;HTTP/1.1 200 Connection Established\r\n\r\n&quot;</span>))</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">done := <span class="built_in">make</span>(<span class="keyword">chan</span> <span class="keyword">struct</span>&#123;&#125;, <span class="number">2</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> &#123;</span><br><span class="line">_, _ = io.Copy(remoteConn, tlsConn)</span><br><span class="line">done &lt;- <span class="keyword">struct</span>&#123;&#125;&#123;&#125;</span><br><span class="line">&#125;()</span><br><span class="line"></span><br><span class="line"><span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> &#123;</span><br><span class="line">_, _ = io.Copy(tlsConn, remoteConn)</span><br><span class="line">done &lt;- <span class="keyword">struct</span>&#123;&#125;&#123;&#125;</span><br><span class="line">&#125;()</span><br><span class="line"></span><br><span class="line">&lt;-done</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>使用了https代理，实际上就是客户端到浏览器使用了TLS握手。但是实际上去Firefox配置<code>127.0.0.1:9999</code>的https代理后依然无法访问，原因是这个签出来的证书<strong>不被信任</strong>。如果有域名，这里推荐使用Let’s Entrypt来给自己的域名进行签发，然后域名再解析到<code>127.0.0.1:9999</code>。</p><p>当然我们也可以自己生成一个，然后导入到浏览器里面，让Firefox<strong>直接信任</strong>这个证书。follow下面的步骤：</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">// CA 私钥，自己保存</span><br><span class="line">&gt; openssl genrsa -out ca.key 4096</span><br><span class="line"></span><br><span class="line">// CA 证书，可以公开</span><br><span class="line">&gt; openssl req -x509 -new -nodes \</span><br><span class="line">  -key ca.key \</span><br><span class="line">  -sha256 \</span><br><span class="line">  -days 3650 \</span><br><span class="line">  -out ca.crt \</span><br><span class="line">  -subj <span class="string">&quot;/CN=Local Dev Proxy CA&quot;</span> \</span><br><span class="line">  -addext <span class="string">&quot;basicConstraints=critical,CA:TRUE,pathlen:0&quot;</span> \</span><br><span class="line">  -addext <span class="string">&quot;keyUsage=critical,keyCertSign,cRLSign&quot;</span> \</span><br><span class="line">  -addext <span class="string">&quot;subjectKeyIdentifier=hash&quot;</span></span><br><span class="line"></span><br><span class="line">// 服务器私钥，自己保存</span><br><span class="line">&gt; openssl genrsa -out server.key 2048</span><br></pre></td></tr></table></figure><p>然后创建server.cnf，写入下面的内容:</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">[req]</span><br><span class="line">distinguished_name = req_distinguished_name</span><br><span class="line">req_extensions = v3_req</span><br><span class="line">prompt = no</span><br><span class="line"></span><br><span class="line">[req_distinguished_name]</span><br><span class="line">CN = localhost</span><br><span class="line"></span><br><span class="line">[v3_req]</span><br><span class="line">basicConstraints = CA:FALSE</span><br><span class="line">keyUsage = critical, digitalSignature, keyEncipherment</span><br><span class="line">extendedKeyUsage = serverAuth</span><br><span class="line">subjectAltName = @alt_names</span><br><span class="line"></span><br><span class="line">[alt_names]</span><br><span class="line">DNS.1 = localhost</span><br><span class="line">IP.1 = 127.0.0.1</span><br></pre></td></tr></table></figure><p>接下来执行命令:</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line">&gt; openssl req -new \</span><br><span class="line">  -key server.key \</span><br><span class="line">  -out server.csr \</span><br><span class="line">  -config server.cnf</span><br><span class="line"></span><br><span class="line">// 用 CA 签发服务端证书，可以公开</span><br><span class="line">&gt; openssl x509 -req \</span><br><span class="line">  -<span class="keyword">in</span> server.csr \</span><br><span class="line">  -CA ca.crt \</span><br><span class="line">  -CAkey ca.key \</span><br><span class="line">  -CAcreateserial \</span><br><span class="line">  -out server.crt \</span><br><span class="line">  -days 825 \</span><br><span class="line">  -sha256 \</span><br><span class="line">  -extensions v3_req \</span><br><span class="line">  -extfile server.cnf</span><br><span class="line"></span><br><span class="line">// 检查证书 SAN</span><br><span class="line">&gt; openssl x509 -<span class="keyword">in</span> server.crt -noout -text | grep -A2 <span class="string">&quot;Subject Alternative Name&quot;</span></span><br><span class="line">X509v3 Subject Alternative Name: </span><br><span class="line">                DNS:localhost, IP Address:127.0.0.1</span><br><span class="line">            X509v3 Subject Key Identifier:</span><br></pre></td></tr></table></figure><p>我们就有了<code>server.crt</code>和<code>server.key</code>，也就是程序里<code>LoadX509KeyPair</code>需要的那两个文件。接下来Firefox需要导入ca.crt文件：</p><p>进入<code>设置-&gt;隐私与安全-&gt;证书-&gt;管理证书-&gt;证书颁发机构-&gt;导入</code>选我们的ca.crt即可。</p><p>接下来<code>代理设置-&gt;配置代理</code>选中<code>自动代理配置的 URL（PAC）w</code>，输入<code>data:,function FindProxyForURL(url, host) { return &quot;HTTPS 127.0.0.1:9999&quot;; }</code>，这代表使用https代理。</p><h1 id="关于证书"><a href="#关于证书" class="headerlink" title="关于证书"></a>关于证书</h1><p>证书可以理解为一个身份认证。想象这样一个场景：证书可以理解为一个身份认证。想象这样一个场景：</p><p>你是一个DNS攻击者，你想要把“淘宝”重定向到你的“黑马电商”狠狠赚一笔。于是用户访问<a href="https://www.taobao.com./">https://www.taobao.com。</a></p><p>正常情况下，DNS应该告诉用户<a href="http://www.taobao.com对应真正淘宝服务器的IP。但你偷偷做了手脚，让很多人的dns请求拿成了你自己的服务器IP：www.taobao.com">www.taobao.com对应真正淘宝服务器的IP。但你偷偷做了手脚，让很多人的dns请求拿成了你自己的服务器IP：www.taobao.com</a> → 你的黑马电商服务器 IP。</p><p>这样用户的浏览器确实会被你带到你的服务器。但是问题来了：浏览器不只是看“我连到了哪个 IP”，它还会问服务器：你怎么证明你就是 <a href="http://www.taobao.com/?%E8%BF%99%E4%B8%AA%E6%97%B6%E5%80%99%EF%BC%8C%E6%9C%8D%E5%8A%A1%E5%99%A8%E9%9C%80%E8%A6%81%E6%8B%BF%E5%87%BA%E4%B8%80%E5%BC%A0%E2%80%9C%E8%AF%81%E4%B9%A6%E2%80%9D%E3%80%82">www.taobao.com?这个时候，服务器需要拿出一张“证书”。</a></p><p>比如淘宝的服务器会拿出一张证书，上面写着：</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">这个证书属于：www.taobao.com</span><br><span class="line">签发机构：某个受信任的 CA</span><br><span class="line">有效期：某年某月某日到某年某月某日</span><br><span class="line">公钥：一串公开的加密材料</span><br><span class="line">签名：CA 对这张证书的签名</span><br></pre></td></tr></table></figure><p>浏览器拿到这张证书后，会检查几件事：</p><ol><li>这张证书是不是可信机构签发的？</li><li>证书有没有过期？</li><li>证书上的域名是不是 <a href="http://www.taobao.com？">www.taobao.com？</a><br> 注：如果浏览器直接访问 <a href="https://xx.xx.xx.xx,则会检查服务端证书里是否包含xx.xx.xx.xx这个/">https://xx.xx.xx.xx，则会检查服务端证书里是否包含xx.xx.xx.xx这个</a> IP 身份，这个不怎么常用。<br> 一般都是给域名签证书，然后域名解析到地址，而不会直接给地址进行签名。</li><li>证书有没有被篡改？</li><li>服务器是否真的拥有这张证书对应的私钥？</li></ol><p>如果这些检查都通过，浏览器才会认为我现在连接的服务器。</p><p>结论：</p><ul><li>域名证书：DNS可以决定用户连接到哪个IP，但证书决定这个服务器能不能证明自己是谁。</li><li>IP证书(不常用)：用户直接访问IP地址时，客户端会验证证书SAN里是否包含这个IP Address身份。</li></ul><h1 id="更优雅的架构"><a href="#更优雅的架构" class="headerlink" title="更优雅的架构"></a>更优雅的架构</h1><p>在当前的http&#x2F;https代理架构下，客户端直连代理服务器，但是更常见的架构是:</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">浏览器--&gt;梯子客户端--&gt;梯子服务端-&gt;目标网站</span><br></pre></td></tr></table></figure><p>这样做的好处是我们可以自己做更多细致的加密，流量控制策略，心跳，重连，连接复用等等功能。并且我们也可以自己做加密，不用所谓的证书了，避免配置https proxy。</p><p>我总结一些非常有意义的优化方向：</p><ol><li>复杂分流：客户端本地筛选流量，对某些网站的访问走直接，某些网站访问走代理。</li><li>协议发现：peek前几个字节，可以知道浏览器使用的协议类型，这样就可以支持单端口多协议。</li><li>连接复用：正如我们在前面keep-alive细节讨论中得到的结论所述，http(s)代理没有解决一条长连接<strong>队头阻塞问题</strong>，也就是一条连接在同一时刻只能处理一个请求。但若自己实现梯子客户端，梯子客户端和服务端可以使用少量长连接，复用它们发送来自浏览器的多个请求，并且服务端也可以并发进行请求，这样不但速度更快，也能节省用户侧到梯子服务端的长连接数量。</li><li>自定义加密：架构是<code>浏览器--http代理协议--&gt;本地代理客户端--自定义加密--&gt;远端代理服务器</code>的架构，其中本地代理客户端到远端代理服务器可以自定义加密隧道协议。且对于之前遇到的Firefox不信任证书的问题也可以解决。</li></ol><h1 id="QUIC协议与证书申请"><a href="#QUIC协议与证书申请" class="headerlink" title="QUIC协议与证书申请"></a>QUIC协议与证书申请</h1><p>梯子客户端到梯子服务端的通讯被称为隧道，这里可以考虑使用QUIC或者TCP TLS。</p><p>QUIC强制要求使用TLS握手，非常安全，且一条QUIC连接支持多路逻辑连接，标准术语叫streams(流)。从设计的角度来说QUIC很适合作为加密隧道。</p><p>为了使用QUIC，我们最好去买一个域名并且用Let’s Entrypt申请一张证书，具体参考这个：<a><a href="https://docs.dnspod.cn/dns/acme-sh/">https://docs.dnspod.cn/dns/acme-sh/</a></a>。</p><p>最后我们能拥下列文件：</p><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ca<span class="selector-class">.cer</span>  fullchain<span class="selector-class">.cer</span>  markity<span class="selector-class">.cn</span><span class="selector-class">.cer</span>  markity<span class="selector-class">.cn</span><span class="selector-class">.conf</span>  markity<span class="selector-class">.cn</span><span class="selector-class">.csr</span>  markity<span class="selector-class">.cn</span><span class="selector-class">.csr</span><span class="selector-class">.conf</span>  markity<span class="selector-class">.cn</span>.key</span><br></pre></td></tr></table></figure><p>使用<code>cert, err := tls.LoadX509KeyPair(&quot;xxx.cer&quot;, &quot;xxx.key&quot;)</code>。</p><h1 id="最终实现"><a href="#最终实现" class="headerlink" title="最终实现"></a>最终实现</h1><p>参见github：<a><a href="https://github.com/markity/crush-proxy">https://github.com/markity/crush-proxy</a></a>，如果是空仓库就是还没写，总有一天会写的。</p>]]>
    </content>
    <id>https://markity.github.io/http-proxy/</id>
    <link href="https://markity.github.io/http-proxy/"/>
    <published>2026-06-02T10:19:00.000Z</published>
    <summary>
      <![CDATA[<p>本文讲解http代理的原理，以及Go如何实现这样一个代理，希望能帮助读者更好地理解平时使用的翻墙工具。</p>]]>
    </summary>
    <title>探索并实现http代理</title>
    <updated>2026-06-02T10:19:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Markity</name>
    </author>
    <content>
      <![CDATA[<p>不同于Go的<code>goroutine</code>和Java 21的<code>virtual threads</code>这类“有栈协程”模型，C#和Python更常使用<code>async/await</code>的“无栈协程”模型来处理并发。本文探索C#的<code>async/await</code>并发模型，以及它与Go有栈协程的差异。</p><span id="more"></span><h1 id="先看几段经典的C-异步代码"><a href="#先看几段经典的C-异步代码" class="headerlink" title="先看几段经典的C#异步代码"></a>先看几段经典的C#异步代码</h1><ol><li>并发请求多个数据源，常用于微服务架构，并发从多个内容服务拉取信息，可以优化响应RT。这个例子展示了如何并发请求多个服务，并用<code>Task.WhenAll</code>等待所有结果：</li></ol><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"><span class="keyword">using</span> System.Diagnostics;</span><br><span class="line"></span><br><span class="line">Stopwatch totalSw = Stopwatch.StartNew();</span><br><span class="line"></span><br><span class="line">Task&lt;HttpResponseMessage&gt; resp1 = GetWithTimingAsync(<span class="string">&quot;https://www.baidu.com&quot;</span>, <span class="string">&quot;百度&quot;</span>);</span><br><span class="line">Task&lt;HttpResponseMessage&gt; resp2 = GetWithTimingAsync(<span class="string">&quot;https://www.qq.com&quot;</span>, <span class="string">&quot;QQ&quot;</span>);</span><br><span class="line">Task&lt;HttpResponseMessage&gt; resp3 = GetWithTimingAsync(<span class="string">&quot;https://bing.com&quot;</span>, <span class="string">&quot;Bing&quot;</span>);</span><br><span class="line"></span><br><span class="line"><span class="keyword">await</span> Task.WhenAll(resp1, resp2, resp3);</span><br><span class="line">totalSw.Stop();</span><br><span class="line">Console.WriteLine(<span class="string">$&quot;总耗时: <span class="subst">&#123;totalSw.ElapsedMilliseconds&#125;</span> ms&quot;</span>);</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">static</span> <span class="keyword">async</span> Task&lt;HttpResponseMessage&gt; <span class="title">GetWithTimingAsync</span>(<span class="params"><span class="built_in">string</span> url, <span class="built_in">string</span> name</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    Stopwatch sw = Stopwatch.StartNew();</span><br><span class="line">    HttpClient client = <span class="keyword">new</span> HttpClient();</span><br><span class="line">    <span class="keyword">try</span></span><br><span class="line">    &#123;</span><br><span class="line">        HttpResponseMessage response = <span class="keyword">await</span> client.GetAsync(url);</span><br><span class="line">        sw.Stop();</span><br><span class="line">        Console.WriteLine(<span class="string">$&quot;<span class="subst">&#123;name&#125;</span> 请求耗时: <span class="subst">&#123;sw.ElapsedMilliseconds&#125;</span> ms, 状态码: <span class="subst">&#123;response.StatusCode&#125;</span>&quot;</span>);</span><br><span class="line">        <span class="keyword">return</span> response;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">catch</span> (Exception ex)</span><br><span class="line">    &#123;</span><br><span class="line">        sw.Stop();</span><br><span class="line">        Console.WriteLine(<span class="string">$&quot;<span class="subst">&#123;name&#125;</span> 请求失败，耗时: <span class="subst">&#123;sw.ElapsedMilliseconds&#125;</span> ms, 异常: <span class="subst">&#123;ex.Message&#125;</span>&quot;</span>);</span><br><span class="line">        <span class="keyword">throw</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">finally</span></span><br><span class="line">    &#123;</span><br><span class="line">        client.Dispose();</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><ol start="2"><li>这个例子展示了一个最基础的TCP Echo Server：</li></ol><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"><span class="keyword">using</span> System.Net;</span><br><span class="line"><span class="keyword">using</span> System.Net.Sockets;</span><br><span class="line"></span><br><span class="line"><span class="keyword">await</span> RunTCPEchoServer();</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">static</span> <span class="keyword">async</span> Task <span class="title">RunTCPEchoServer</span>()</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">var</span> listener = <span class="keyword">new</span> TcpListener(IPAddress.Loopback, <span class="number">5000</span>);</span><br><span class="line">    listener.Start();</span><br><span class="line">    Console.WriteLine(<span class="string">&quot;Server started on 127.0.0.1:5000&quot;</span>);</span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>)</span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">var</span> client = <span class="keyword">await</span> listener.AcceptTcpClientAsync();</span><br><span class="line"></span><br><span class="line">        _ = HandleClient(client);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">static</span> <span class="keyword">async</span> Task <span class="title">HandleClient</span>(<span class="params">TcpClient client</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">using</span> <span class="keyword">var</span> c = client;</span><br><span class="line">    <span class="keyword">var</span> stream = c.GetStream();</span><br><span class="line"></span><br><span class="line">    <span class="keyword">var</span> buffer = <span class="keyword">new</span> <span class="built_in">byte</span>[<span class="number">1024</span>];</span><br><span class="line"></span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>)</span><br><span class="line">    &#123;</span><br><span class="line">        <span class="built_in">int</span> n = <span class="keyword">await</span> stream.ReadAsync(buffer);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">if</span> (n == <span class="number">0</span>)</span><br><span class="line">            <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line">        <span class="keyword">await</span> stream.WriteAsync(buffer.AsMemory(<span class="number">0</span>, n));</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><ol start="3"><li>这个例子展示了后台任务<code>fire-and-forget</code>的写法：</li></ol><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">_ = RunMyIntervalTaskAsync(); <span class="comment">// just ignore Task handle</span></span><br><span class="line"></span><br><span class="line"><span class="comment">// wait forever</span></span><br><span class="line">TaskCompletionSource src = <span class="keyword">new</span> TaskCompletionSource();</span><br><span class="line"><span class="keyword">await</span> src.Task;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">async</span> <span class="keyword">static</span> Task <span class="title">RunMyIntervalTaskAsync</span>()</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>)</span><br><span class="line">    &#123;</span><br><span class="line">        Console.WriteLine(<span class="string">&quot;Do task&quot;</span>);</span><br><span class="line">        <span class="keyword">await</span> Task.Delay(<span class="number">3000</span>);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p><strong>总结:</strong></p><ul><li>命名规约：要给<code>async</code>修饰的异步函数名加上<code>Async</code>后缀。</li><li>调用异步函数给人的感觉是启动了一个异步流程，可以同时不阻塞地启动多个这样的流程。</li><li>async方法调用后会先同步执行，直到遇到第一个未完成的await才返回给调用方。更准确说法是：调用返回一个代表异步操作最终结果的Task，该操作可能已经开始，也可能已经完成，具体取决于方法实现。</li><li>如果想要 <code>fire and forget</code>，类似Go启动一个背景goroutine，直接<code>_ = XxxAsync()</code>抛掉<code>awaitable</code>句柄即可。生产上真正使用时要捕获异常、接入日志、取消令牌，服务端长期后台任务优先用BackgroundService&#x2F;队列消费者。</li></ul><h1 id="Go有栈协程序概述"><a href="#Go有栈协程序概述" class="headerlink" title="Go有栈协程序概述"></a>Go有栈协程序概述</h1><p>看完上面的代码，可以知道无栈协程的使用相当简单，但从概念以及原理上来说它比有栈协程复杂很多。</p><p>使用Go协程，没有<code>Task&lt;&gt;</code>，也没有<code>async/await</code>这类关键字；启动一个协程就像启动一个线程一样自然；之后的Read、Write socket也全部都是同步API，用起来就像在C语言里使用线程 + 阻塞IO一样简单自然。自然可见，Go的协程模型比C#简单很多。</p><p>Go协程简单来说就是语言把每一个执行流抽象成了一个goroutine，也就是协程，这个goroutine里面有一条执行流的所有上下文信息，包括寄存器、当前执行指令的位置、栈等等信息。</p><p>实际上，main函数的那条执行流也是一个goroutine，从main goroutine可以创建其它goroutine，下面是一个简单的go echo server:</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line"><span class="string">&quot;bufio&quot;</span></span><br><span class="line"><span class="string">&quot;fmt&quot;</span></span><br><span class="line"><span class="string">&quot;io&quot;</span></span><br><span class="line"><span class="string">&quot;log&quot;</span></span><br><span class="line"><span class="string">&quot;net&quot;</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> &#123;</span><br><span class="line">listener, err := net.Listen(<span class="string">&quot;tcp&quot;</span>, <span class="string">&quot;:9000&quot;</span>)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">log.Fatal(err)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">fmt.Println(<span class="string">&quot;Echo server listening on :9000&quot;</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> &#123;</span><br><span class="line">conn, err := listener.Accept()</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">log.Println(<span class="string">&quot;accept error:&quot;</span>, err)</span><br><span class="line"><span class="keyword">continue</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">go</span> handleConn(conn)</span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">handleConn</span><span class="params">(conn net.Conn)</span></span> &#123;</span><br><span class="line"><span class="keyword">defer</span> conn.Close()</span><br><span class="line"></span><br><span class="line">addr := conn.RemoteAddr().String()</span><br><span class="line">fmt.Println(<span class="string">&quot;client connected:&quot;</span>, addr)</span><br><span class="line"></span><br><span class="line">reader := bufio.NewReader(conn)</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> &#123;</span><br><span class="line">line, err := reader.ReadString(<span class="string">&#x27;\n&#x27;</span>)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">if</span> err != io.EOF &#123;</span><br><span class="line">log.Println(<span class="string">&quot;read error:&quot;</span>, err)</span><br><span class="line">&#125;</span><br><span class="line">fmt.Println(<span class="string">&quot;client disconnected:&quot;</span>, addr)</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">_, err = conn.Write([]<span class="type">byte</span>(line))</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">log.Println(<span class="string">&quot;write error:&quot;</span>, err)</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>可见，使用goroutine就像使用线程一样自然，非常好用。</p><p>Go的并发模型是GMP模型。进程启动的时候会启动多个OS线程（被称为M，Machine的缩写），每个线程只有绑定调度器(被称为P，Processor的缩写)后才能调度协程(被称为G, Goroutine的缩写)。调度器的代码抽象可以理解为不停地取状态为runnable的goroutine进行执行。每次启动一个goroutine都创建了一个“调度体”对象，也就是goroutine。</p><p>当某个goroutine阻塞在某个read socket上时，这时其实并没有阻塞线程(Golang的网络通常会接入runtime netpoll，所以network fd都是用的non-blocking模式)。fd此时被挂在了epoll树上并且该goroutine被runtime park挂起，等到这个socket真正可read的时候，goroutine才恢复runnable状态，被“调度器”重新拿出来跑。</p><p>总结来说，这里有几个关键的名词需要理解：</p><ul><li>调度器：理解为一个不断poll runnable goroutine的for循环，抢goroutine运行。</li><li>调度体&#x2F;goroutine：保存goroutine上下文，这就是一条goroutine执行流的抽象。</li><li>epoller(仅限linux): 实现阻塞挂起，Go的runtime会把阻塞的goroutine挂起，等事件到来后再恢复。例如当read socket读到空（non-blocking模式返回EAGAIN）时，就把当前goroutine的状态改为挂起，并且在epoll上注册这个socket。</li><li>有栈协程：保存了执行流完整的栈信息的协程。</li></ul><p>关于 epoll 挂起，这里有一些代码可以加以佐证：</p><figure class="highlight golang"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(fd *FD)</span></span> Read(p []<span class="type">byte</span>) (<span class="type">int</span>, <span class="type">error</span>) &#123;</span><br><span class="line"><span class="keyword">if</span> err := fd.readLock(); err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">return</span> <span class="number">0</span>, err</span><br><span class="line">&#125;</span><br><span class="line"><span class="keyword">defer</span> fd.readUnlock()</span><br><span class="line"><span class="keyword">if</span> <span class="built_in">len</span>(p) == <span class="number">0</span> &#123;</span><br><span class="line"><span class="comment">// If the caller wanted a zero byte read, return immediately</span></span><br><span class="line"><span class="comment">// without trying (but after acquiring the readLock).</span></span><br><span class="line"><span class="comment">// Otherwise syscall.Read returns 0, nil which looks like</span></span><br><span class="line"><span class="comment">// io.EOF.</span></span><br><span class="line"><span class="comment">// TODO(bradfitz): make it wait for readability? (Issue 15735)</span></span><br><span class="line"><span class="keyword">return</span> <span class="number">0</span>, <span class="literal">nil</span></span><br><span class="line">&#125;</span><br><span class="line"><span class="keyword">if</span> err := fd.pd.prepareRead(fd.isFile); err != <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">return</span> <span class="number">0</span>, err</span><br><span class="line">&#125;</span><br><span class="line"><span class="keyword">if</span> fd.IsStream &amp;&amp; <span class="built_in">len</span>(p) &gt; maxRW &#123;</span><br><span class="line">p = p[:maxRW]</span><br><span class="line">&#125;</span><br><span class="line"><span class="keyword">for</span> &#123;</span><br><span class="line">n, err := ignoringEINTRIO(syscall.Read, fd.Sysfd, p)</span><br><span class="line"><span class="keyword">if</span> err != <span class="literal">nil</span> &#123;</span><br><span class="line">n = <span class="number">0</span></span><br><span class="line"><span class="keyword">if</span> err == syscall.EAGAIN &amp;&amp; fd.pd.pollable() &#123;</span><br><span class="line"><span class="keyword">if</span> err = fd.pd.waitRead(fd.isFile); err == <span class="literal">nil</span> &#123;</span><br><span class="line"><span class="keyword">continue</span></span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br><span class="line">err = fd.eofError(n, err)</span><br><span class="line"><span class="keyword">return</span> n, err</span><br><span class="line">&#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>此处waitRead就是把goroutine挂起到epoll上的相关逻辑，将fd的readable事件注册，并且挂起goroutine。</p><h1 id="C-无栈协程是怎么实现的"><a href="#C-无栈协程是怎么实现的" class="headerlink" title="C#无栈协程是怎么实现的"></a>C#无栈协程是怎么实现的</h1><p>结合我前面提到的几个例子很快就能理解C#异步的用法，这里我讲讲它的实现原理。</p><p>C#的async&#x2F;await并不是靠”真·线程切换”来实现协程，而是通过编译器生成状态机，在await处将自身注册到所等待的awaiter上，保存执行上下文。在任务完成时，执行恢复回调，继续之前的流程。</p><p>下面先看Task究竟是什么，见下面的代码：</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title">Program</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">async</span> <span class="keyword">static</span> Task <span class="title">Main</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="built_in">int</span> val = <span class="keyword">await</span> GetNumberAsync();</span><br><span class="line">        Console.WriteLine(val);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">static</span> Task&lt;<span class="built_in">int</span>&gt; <span class="title">GetNumberAsync</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        TaskCompletionSource&lt;<span class="built_in">int</span>&gt; src = <span class="keyword">new</span>();</span><br><span class="line">        Thread thread = <span class="keyword">new</span> Thread(() =&gt;</span><br><span class="line">        &#123;</span><br><span class="line">            Thread.Sleep(<span class="number">3000</span>);</span><br><span class="line">            src.SetResult(<span class="number">1</span>);</span><br><span class="line">        &#125;)</span><br><span class="line">        &#123; IsBackground = <span class="literal">true</span> &#125;;</span><br><span class="line">        thread.Start();</span><br><span class="line">        <span class="keyword">return</span> src.Task;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">// 运行结果: 3s后输出1</span></span><br></pre></td></tr></table></figure><p>可以看到，只要给Task SetResult，之前await它的执行流就会恢复了，我们可以直接使用<code>TaskCompletionSource</code>来创建一个source。</p><p>实际上，我们平时使用的很多 <code>Task.Delay()</code>、<code>ReadAsync()</code> 等方法，底层都会返回类似这样的Task给调用方。调用方<code>await</code>它时，执行流会挂起；等Task完成后，再恢复执行。</p><p>接下来我们来看async函数有什么“魔法”，实际上，async函数会被编译器直接翻译成一个“状态机”，它类似这样：</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">static</span> <span class="keyword">async</span> Task&lt;<span class="built_in">int</span>&gt; <span class="title">FooAsync</span>()</span></span><br><span class="line">&#123;</span><br><span class="line">    Console.WriteLine(<span class="string">&quot;A&quot;</span>);</span><br><span class="line"></span><br><span class="line">    <span class="built_in">int</span> x = <span class="keyword">await</span> GetNumberAsync();</span><br><span class="line"></span><br><span class="line">    Console.WriteLine(<span class="string">&quot;B&quot;</span>);</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> x + <span class="number">1</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">static</span> Task&lt;<span class="built_in">int</span>&gt; <span class="title">GetNumberAsync</span>()</span></span><br><span class="line">&#123;</span><br><span class="line">    TaskCompletionSource&lt;<span class="built_in">int</span>&gt; src = <span class="keyword">new</span>();</span><br><span class="line">    Thread thread = <span class="keyword">new</span> Thread(() =&gt;</span><br><span class="line">    &#123;</span><br><span class="line">        Thread.Sleep(<span class="number">3000</span>);</span><br><span class="line">        src.SetResult(<span class="number">1</span>);</span><br><span class="line">    &#125;)</span><br><span class="line">    &#123; IsBackground = <span class="literal">true</span> &#125;;</span><br><span class="line">    thread.Start();</span><br><span class="line">    <span class="keyword">return</span> src.Task;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 上面是用户代码，下面是编译器伪代码，看个意思，代码并不严谨</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">static</span> Task&lt;<span class="built_in">int</span>&gt; <span class="title">FooAsyncAfterInterpreted</span>()</span></span><br><span class="line">&#123;</span><br><span class="line">    FooAsyncStateMachine stateMachine = <span class="keyword">new</span> FooAsyncStateMachine();</span><br><span class="line"></span><br><span class="line">    stateMachine.builder = AsyncTaskMethodBuilder&lt;<span class="built_in">int</span>&gt;.Create();</span><br><span class="line">    <span class="comment">// 初始值-1</span></span><br><span class="line">    stateMachine.state = <span class="number">-1</span>;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 执行一次MoveNext</span></span><br><span class="line">    stateMachine.builder.Start(<span class="keyword">ref</span> stateMachine);</span><br><span class="line"></span><br><span class="line">    <span class="keyword">return</span> stateMachine.builder.Task;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">private</span> <span class="keyword">struct</span> FooAsyncStateMachine : IAsyncStateMachine</span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">public</span> <span class="built_in">int</span> state;</span><br><span class="line">    <span class="keyword">public</span> AsyncTaskMethodBuilder&lt;<span class="built_in">int</span>&gt; builder;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 会把局部变量的上下文存储在状态机里面</span></span><br><span class="line">    <span class="keyword">private</span> <span class="built_in">int</span> x;</span><br><span class="line">    <span class="keyword">private</span> TaskAwaiter&lt;<span class="built_in">int</span>&gt; awaiter;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">MoveNext</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="built_in">int</span> result;</span><br><span class="line"></span><br><span class="line">        <span class="keyword">try</span></span><br><span class="line">        &#123;</span><br><span class="line">            TaskAwaiter&lt;<span class="built_in">int</span>&gt; localAwaiter;</span><br><span class="line">            <span class="comment">// state == -1，执行第一个await之前的代码</span></span><br><span class="line">            <span class="keyword">if</span> (state == <span class="number">-1</span>)</span><br><span class="line">            &#123;</span><br><span class="line">                Console.WriteLine(<span class="string">&quot;A&quot;</span>);</span><br><span class="line"></span><br><span class="line">                Task&lt;<span class="built_in">int</span>&gt; task = GetNumberAsync();</span><br><span class="line">                localAwaiter = task.GetAwaiter();</span><br><span class="line"></span><br><span class="line">                <span class="keyword">if</span> (!localAwaiter.IsCompleted)</span><br><span class="line">                &#123;</span><br><span class="line">                    <span class="comment">// state == 0: 等待第一个await中</span></span><br><span class="line">                    state = <span class="number">0</span>;</span><br><span class="line">                    awaiter = localAwaiter;</span><br><span class="line"></span><br><span class="line">                    builder.AwaitUnsafeOnCompleted(</span><br><span class="line">                        <span class="keyword">ref</span> localAwaiter,</span><br><span class="line">                        <span class="keyword">ref</span> <span class="keyword">this</span>);</span><br><span class="line"></span><br><span class="line">                    <span class="keyword">return</span>;</span><br><span class="line">                &#125;</span><br><span class="line">            &#125;</span><br><span class="line">            <span class="comment">// 从state == 0，也就是等待第一个await中恢复过来</span></span><br><span class="line">            <span class="keyword">else</span> <span class="keyword">if</span> (state == <span class="number">0</span>)</span><br><span class="line">            &#123;</span><br><span class="line">                localAwaiter = awaiter;</span><br><span class="line">                awaiter = <span class="literal">default</span>;</span><br><span class="line">                state = <span class="number">-1</span>;</span><br><span class="line">            &#125;</span><br><span class="line"></span><br><span class="line">            x = localAwaiter.GetResult();</span><br><span class="line"></span><br><span class="line">            Console.WriteLine(<span class="string">&quot;B&quot;</span>);</span><br><span class="line"></span><br><span class="line">            result = x + <span class="number">1</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">// state=-2 -&gt; 已完成</span></span><br><span class="line">        <span class="keyword">catch</span> (Exception ex)</span><br><span class="line">        &#123;</span><br><span class="line">            state = <span class="number">-2</span>;</span><br><span class="line">            builder.SetException(ex);</span><br><span class="line">            <span class="keyword">return</span>;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        state = <span class="number">-2</span>;</span><br><span class="line">        builder.SetResult(result);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">SetStateMachine</span>(<span class="params">IAsyncStateMachine stateMachine</span>)</span></span><br><span class="line">    &#123;</span><br><span class="line">        builder.SetStateMachine(stateMachine);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>下面总结几个关键点：</p><ul><li>FooAsync被编译器翻译成了一个返回Task的普通函数，并且真实运行逻辑以及上下文被打包进了一个状态机struct。</li><li>FooAsync的核心逻辑在StateMachine的MoveNext中，state定位当前运行状态(await位置以及是否执行完毕)。</li><li>异步调用的局部变量上下文被保存在了StateMachine对象当中。</li><li>await处编译器做了改动，检查awaiter是否完成，没有完成就把continuation(指接下来要继续执行的那段代码)注册到对应的Task上去。底层Task完成后，触发之前注册的continuation，也就是调用自己列表里面所有continuation的MoveNext。</li></ul><h1 id="C-线程模型与异步的关系"><a href="#C-线程模型与异步的关系" class="headerlink" title="C#线程模型与异步的关系"></a>C#线程模型与异步的关系</h1><p>创建<code>TaskCompletionSource</code>可以使用(几乎总是)一个参数<code>RunContinuationsAsynchronously</code>。观察下面的代码：</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title">Program</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">async</span> <span class="keyword">static</span> Task <span class="title">Main</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        Console.WriteLine(<span class="string">$&quot;1: <span class="subst">&#123;Environment.CurrentManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line">        <span class="built_in">int</span> val = <span class="keyword">await</span> GetNumberAsync();</span><br><span class="line">        Console.WriteLine(<span class="string">$&quot;2: <span class="subst">&#123;Environment.CurrentManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">static</span> Task&lt;<span class="built_in">int</span>&gt; <span class="title">GetNumberAsync</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        Console.WriteLine(<span class="string">$&quot;3: <span class="subst">&#123;Environment.CurrentManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line">        TaskCompletionSource&lt;<span class="built_in">int</span>&gt; src = <span class="keyword">new</span>(TaskCreationOptions.RunContinuationsAsynchronously);</span><br><span class="line">        Thread thread = <span class="keyword">new</span> Thread(() =&gt;</span><br><span class="line">        &#123;</span><br><span class="line">            Console.WriteLine(<span class="string">$&quot;4: <span class="subst">&#123;Environment.CurrentManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line">            Thread.Sleep(<span class="number">3000</span>);</span><br><span class="line">            src.SetResult(<span class="number">1</span>);</span><br><span class="line">        &#125;)</span><br><span class="line">        &#123; IsBackground = <span class="literal">true</span> &#125;;</span><br><span class="line">        thread.Start();</span><br><span class="line">        <span class="keyword">return</span> src.Task;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"><span class="comment">// 输出：</span></span><br><span class="line"><span class="comment">// 1: 1</span></span><br><span class="line"><span class="comment">// 3: 1</span></span><br><span class="line"><span class="comment">// 4: 4</span></span><br><span class="line"><span class="comment">// 2: 5</span></span><br><span class="line"></span><br><span class="line"><span class="comment">// 当不使用RunContinuationsAsynchronously参数的时候输出：</span></span><br><span class="line"><span class="comment">// 1: 1</span></span><br><span class="line"><span class="comment">// 3: 1</span></span><br><span class="line"><span class="comment">// 4: 4</span></span><br><span class="line"><span class="comment">// 2: 4</span></span><br></pre></td></tr></table></figure><p>可以看到await结束，之前运行在thread1的执行流实际上换到了thread5执行，若不指定参数，执行流可能(受SynchronizationContext和非默认TaskScheduler的影响，这点以后再说)直接在SetResult线程恢复执行。</p><p>若指定了<code>RunContinuationsAsynchronously</code>参数，恢复continuation具体流程如下：</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">await 发现 Task 尚未完成，状态机注册 continuation(若Task已经完成则在当前线程继续跑)</span><br><span class="line">        ↓</span><br><span class="line">TaskCompletionSource.SetResult 完成 Task</span><br><span class="line">        ↓</span><br><span class="line">如果有 RunContinuationsAsynchronously，Task 完成时会要求 continuation 不要同步内联运行在当前调用栈里，而是投递到对应调度器</span><br><span class="line">        ↓</span><br><span class="line">continuation 被投递给调度器</span><br><span class="line">        ↓</span><br><span class="line">调度器通常是 ThreadPool / UI SynchronizationContext / 自定义 TaskScheduler</span><br><span class="line">        ↓</span><br><span class="line">某个线程取出 continuation</span><br><span class="line">        ↓</span><br><span class="line">调用状态机 MoveNext()</span><br></pre></td></tr></table></figure><p>典型的异步 I&#x2F;O也通常倾向于避免在底层I&#x2F;O完成回调里直接执行过重的用户continuation，否则 I&#x2F;O 完成路径的耗时会变得不可控。因此有可能，你await一个异步IO回来，发现执行流在别的线程上了。需要注意的是这些 API 底层不一定是通过<code>TaskCompletionSource</code> + <code>RunContinuationsAsynchronously</code>实现的，底层可能是别的东西，但我这里就不感知了，但想必是类似的。</p><p>在普通 Console&#x2F;服务端场景中，异步 I&#x2F;O 完成后的 continuation 通常会被调度到.NET ThreadPool上执行。ThreadPool也有本地队列、全局队列和work stealing机制，这一点和Go调度器中的work stealing思路有相似之处。</p><p>在写一般的异步代码时，实际上是用的多线程，下面是echo server静默时候的线程情况(<code>ps -T -p pid</code>)：</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">dotnet              // 父进程</span><br><span class="line">.NET Finalizer      // GC finalizer 线程</span><br><span class="line">.NET SigHandler     // Linux 信号处理线程</span><br><span class="line">.NET Sockets        // epoll挂起相关</span><br><span class="line">.NET Timer          // Timer相关</span><br><span class="line">.NET TP Wait        // ThreadPool 的等待/注册等待相关线程</span><br></pre></td></tr></table></figure><p>此刻有一个预备的线程，若某个Task完成进入MoveNext代码的时候，也是此线程poll到了MoveNext任务进行执行。当任务的队列开始积压的时候就会建立更多Thread来处理，有动态扩&#x2F;缩容策略，下面是代码中CPU较为密集(CPU Bound)的时候，ThreadPool消费速率不足时的线程状态：</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">dotnet</span><br><span class="line">.NET Finalizer</span><br><span class="line">.NET SigHandler</span><br><span class="line">.NET TP Worker</span><br><span class="line">.NET TP Gate        // 线程池的调度控制器，专门管 “什么时候加线程、加多少、要不要缩容”</span><br><span class="line">.NET TP Worker</span><br><span class="line">.NET TP Worker</span><br><span class="line">.NET Sockets</span><br><span class="line">.NET Timer</span><br><span class="line">.NET TP Wait        // 还在等任务，线程池工作线程处于等待状态</span><br></pre></td></tr></table></figure><h1 id="CPU-Bound-IO-Bound"><a href="#CPU-Bound-IO-Bound" class="headerlink" title="CPU Bound&#x2F;IO Bound"></a>CPU Bound&#x2F;IO Bound</h1><p>CPU Bound的意思是消耗大量CPU时间的<strong>计算型任务</strong>，如：</p><ul><li>大量数学计算</li><li>耗时的加密解密</li><li>图像处理</li></ul><p>IO Bound的意思是不怎么消耗CPU时间，主要是等待IO的<strong>等待外部资源返回数据的任务</strong>，如:</p><ul><li>网络</li><li>磁盘</li><li>数据库访问</li><li>用户&#x2F;设备输入</li></ul><p>观察到上述C#的协程是被运行在ThreadPool中的，也就是当Task完成时，continuation被投递到ThreadPool中运行(此处暂时就这么认为，后续介绍上下文和调度器会推翻这个简单的假设)。新手常见的错误是在async函数里面运行长时间消耗CPU时间的任务，例如大量计算，加密，图像处理等等，下面是两个负面例子：</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">async</span> Task <span class="title">BadHandleClientAsync1</span>(<span class="params">TcpClient client</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">using</span> <span class="keyword">var</span> stream = client.GetStream();</span><br><span class="line"></span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>)</span><br><span class="line">    &#123;</span><br><span class="line">        <span class="built_in">byte</span>[] requestBytes = <span class="keyword">await</span> ReadFrameAsync(stream);</span><br><span class="line"></span><br><span class="line">        <span class="built_in">byte</span>[] decrypted = Decrypt(requestBytes);</span><br><span class="line">        <span class="built_in">byte</span>[] decompressed = Decompress(decrypted);</span><br><span class="line">        Request request = DeserializeLargeJson(decompressed);</span><br><span class="line">        <span class="comment">// CPU-heavy: 复杂业务计算，里面可能有非常复杂的数学计算，图像处理，耗时很长</span></span><br><span class="line">        Response response = RunComplexBusinessRules(request);</span><br><span class="line">        <span class="built_in">byte</span>[] responseJson = SerializeLargeJson(response);</span><br><span class="line">        <span class="built_in">byte</span>[] compressed = Compress(responseJson);</span><br><span class="line">        <span class="built_in">byte</span>[] encrypted = Encrypt(compressed);</span><br><span class="line">        <span class="built_in">byte</span>[] responseBytes = BuildFrame(encrypted);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">await</span> stream.WriteAsync(responseBytes, <span class="number">0</span>, responseBytes.Length);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">async</span> Task <span class="title">BadHandleClientAsync2</span>(<span class="params">TcpClient client</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">using</span> <span class="keyword">var</span> stream = client.GetStream();</span><br><span class="line"></span><br><span class="line">    <span class="keyword">while</span> (<span class="literal">true</span>)</span><br><span class="line">    &#123;</span><br><span class="line">        <span class="built_in">byte</span>[] requestBytes = <span class="keyword">await</span> ReadFrameAsync(stream);</span><br><span class="line"></span><br><span class="line">        <span class="built_in">byte</span>[] responseBytes = <span class="keyword">await</span> Task.Run(() =&gt;</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="built_in">byte</span>[] decrypted = Decrypt(requestBytes);</span><br><span class="line">            <span class="built_in">byte</span>[] decompressed = Decompress(decrypted);</span><br><span class="line">            Request request = DeserializeLargeJson(decompressed);</span><br><span class="line">            Response response = RunComplexBusinessRules(request);</span><br><span class="line">            <span class="built_in">byte</span>[] responseJson = SerializeLargeJson(response);</span><br><span class="line">            <span class="built_in">byte</span>[] compressed = Compress(responseJson);</span><br><span class="line">            <span class="built_in">byte</span>[] encrypted = Encrypt(compressed);</span><br><span class="line">            <span class="keyword">return</span> BuildFrame(encrypted);</span><br><span class="line">        &#125;);</span><br><span class="line">        <span class="keyword">await</span> stream.WriteAsync(responseBytes, <span class="number">0</span>, responseBytes.Length);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>在极端的高并发场景下，上面两个例子，不论是否使用<code>Task.Run</code>都不太是正确的选择，原因如下：</p><ul><li>关于例子1，协程是在ThreadPool线程里运行的，如果在里面执行CPU耗时操作，其实会把这个线程给占住，导致该线程没法消费处理其它已恢复的continuation。当所有线程被用完的时候(均处于CPU计算而阻塞)，Gate线程就会创建新的线程来跑这个continuation。因此有很大的创建过量线程的风险。</li><li>关于例子2，<code>Task.Run</code>的语义是把任务投递给ThreadPool，也是同样的问题，如果线程池里面的任务长时间占据线程，也会面临Gate启动过多线程的问题。</li></ul><p>那么到底怎么解决呢？首先明确一个问题，对于CPU密集任务<strong>增加线程数目不能提高执行效率</strong>，一个核心同时只能跑一个执行流，所以可以考虑将IO Threads和Worker Threads分开，其中Worker Threads的数量为CPU核心数。下面为一个典型的CPU&#x2F;IO分离实现思路：</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">using</span> System.Collections.Concurrent;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">class</span> <span class="title">Program</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">async</span> Task <span class="title">Main</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">var</span> cpuQueue = <span class="keyword">new</span> CpuWorkQueue(</span><br><span class="line">            workerCount: <span class="number">2</span>,</span><br><span class="line">            capacity: <span class="number">10</span>,</span><br><span class="line">            name: <span class="string">&quot;dedicated-cpu-worker&quot;</span>);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">var</span> tasks = <span class="keyword">new</span> List&lt;Task&lt;<span class="built_in">int</span>&gt;&gt;();</span><br><span class="line"></span><br><span class="line">        Console.WriteLine(<span class="string">$&quot;Main thread: <span class="subst">&#123;Thread.CurrentThread.ManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line">        Console.WriteLine(<span class="string">&quot;Start enqueue CPU tasks...&quot;</span>);</span><br><span class="line">        Console.WriteLine();</span><br><span class="line"></span><br><span class="line">        <span class="keyword">for</span> (<span class="built_in">int</span> i = <span class="number">0</span>; i &lt; <span class="number">6</span>; i++)</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="built_in">int</span> taskId = i;</span><br><span class="line"></span><br><span class="line">            Task&lt;<span class="built_in">int</span>&gt; task = cpuQueue.Enqueue(() =&gt;</span><br><span class="line">            &#123;</span><br><span class="line">                Console.WriteLine(</span><br><span class="line">                    <span class="string">$&quot;Task <span class="subst">&#123;taskId&#125;</span> started on thread <span class="subst">&#123;Thread.CurrentThread.ManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line"></span><br><span class="line">                <span class="comment">// 模拟 CPU 密集型计算</span></span><br><span class="line">                <span class="keyword">var</span> result = HeavyCpuCompute(taskId);</span><br><span class="line"></span><br><span class="line">                Console.WriteLine(</span><br><span class="line">                    <span class="string">$&quot;Task <span class="subst">&#123;taskId&#125;</span> finished on thread <span class="subst">&#123;Thread.CurrentThread.ManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line"></span><br><span class="line">                <span class="keyword">return</span> result;</span><br><span class="line">            &#125;);</span><br><span class="line"></span><br><span class="line">            tasks.Add(task);</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        Console.WriteLine(<span class="string">&quot;All tasks have been enqueued.&quot;</span>);</span><br><span class="line">        Console.WriteLine(<span class="string">&quot;Main thread is not blocked while CPU workers are running.&quot;</span>);</span><br><span class="line">        Console.WriteLine();</span><br><span class="line"></span><br><span class="line">        <span class="built_in">int</span>[] results = <span class="keyword">await</span> Task.WhenAll(tasks);</span><br><span class="line"></span><br><span class="line">        Console.WriteLine();</span><br><span class="line">        Console.WriteLine(<span class="string">&quot;All CPU tasks finished.&quot;</span>);</span><br><span class="line">        Console.WriteLine(<span class="string">&quot;Results:&quot;</span>);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">foreach</span> (<span class="keyword">var</span> result <span class="keyword">in</span> results)</span><br><span class="line">        &#123;</span><br><span class="line">            Console.WriteLine(result);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">private</span> <span class="keyword">static</span> <span class="built_in">int</span> <span class="title">HeavyCpuCompute</span>(<span class="params"><span class="built_in">int</span> input</span>)</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="comment">// 真实场景里这里通常是复杂计算、压缩、加密、图像处理、规则引擎等。</span></span><br><span class="line">        <span class="comment">// Sleep这里只是演示工作队列调度效果</span></span><br><span class="line">        Thread.Sleep(<span class="number">2000</span>);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">return</span> input * input;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">sealed</span> <span class="keyword">class</span> <span class="title">CpuWorkQueue</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">private</span> <span class="keyword">readonly</span> BlockingCollection&lt;Action&gt; _queue;</span><br><span class="line">    <span class="keyword">private</span> <span class="keyword">readonly</span> Thread[] _workers;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="title">CpuWorkQueue</span>(<span class="params"><span class="built_in">int</span> workerCount, <span class="built_in">int</span> capacity = <span class="number">1024</span>, <span class="built_in">string</span> name = <span class="string">&quot;cpu-worker&quot;</span></span>)</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">if</span> (workerCount &lt;= <span class="number">0</span>)</span><br><span class="line">            <span class="keyword">throw</span> <span class="keyword">new</span> ArgumentOutOfRangeException(<span class="keyword">nameof</span>(workerCount));</span><br><span class="line"></span><br><span class="line">        <span class="keyword">if</span> (capacity &lt;= <span class="number">0</span>)</span><br><span class="line">            <span class="keyword">throw</span> <span class="keyword">new</span> ArgumentOutOfRangeException(<span class="keyword">nameof</span>(capacity));</span><br><span class="line"></span><br><span class="line">        _queue = <span class="keyword">new</span> BlockingCollection&lt;Action&gt;(boundedCapacity: capacity);</span><br><span class="line">        _workers = <span class="keyword">new</span> Thread[workerCount];</span><br><span class="line"></span><br><span class="line">        <span class="keyword">for</span> (<span class="built_in">int</span> i = <span class="number">0</span>; i &lt; workerCount; i++)</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="keyword">var</span> index = i;</span><br><span class="line"></span><br><span class="line">            _workers[i] = <span class="keyword">new</span> Thread(WorkerLoop)</span><br><span class="line">            &#123;</span><br><span class="line">                IsBackground = <span class="literal">true</span>,</span><br><span class="line">                Name = <span class="string">$&quot;<span class="subst">&#123;name&#125;</span>-<span class="subst">&#123;index&#125;</span>&quot;</span></span><br><span class="line">            &#125;;</span><br><span class="line"></span><br><span class="line">            _workers[i].Start();</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="title">Task</span>&lt;<span class="title">T</span>&gt; <span class="title">Enqueue</span>&lt;<span class="title">T</span>&gt;(<span class="params">Func&lt;T&gt; work</span>)</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">var</span> tcs = <span class="keyword">new</span> TaskCompletionSource&lt;T&gt;(</span><br><span class="line">            TaskCreationOptions.RunContinuationsAsynchronously);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">var</span> item = <span class="keyword">new</span> Action(() =&gt;</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="keyword">try</span></span><br><span class="line">            &#123;</span><br><span class="line">                <span class="keyword">var</span> result = work();</span><br><span class="line">                tcs.TrySetResult(result);</span><br><span class="line">            &#125;</span><br><span class="line">            <span class="keyword">catch</span> (Exception ex)</span><br><span class="line">            &#123;</span><br><span class="line">                tcs.TrySetException(ex);</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;);</span><br><span class="line"></span><br><span class="line">        <span class="comment">// 不要无限排队。队列满了就明确失败，形成背压。</span></span><br><span class="line">        <span class="keyword">if</span> (!_queue.TryAdd(item))</span><br><span class="line">        &#123;</span><br><span class="line">            tcs.TrySetException(</span><br><span class="line">                <span class="keyword">new</span> InvalidOperationException(<span class="string">&quot;CPU work queue is full.&quot;</span>));</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="keyword">return</span> tcs.Task;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">WorkerLoop</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">foreach</span> (<span class="keyword">var</span> work <span class="keyword">in</span> _queue.GetConsumingEnumerable())</span><br><span class="line">        &#123;</span><br><span class="line">            work();</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h1 id="协程调度"><a href="#协程调度" class="headerlink" title="协程调度"></a>协程调度</h1><p>上面提到了<code>SynchronizationContext</code>和<code>TaskScheduler</code>，它们不但难以理解而且还很重要。它们直接决定了await Task结束后，此协程在哪里恢复运行。<strong>这里有个调度的优先级规则</strong>，如下：</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">如果 SynchronizationContext.Current != null</span><br><span class="line">    用 SynchronizationContext.Post(...) 调度 continuation</span><br><span class="line"></span><br><span class="line">否则如果 TaskScheduler.Current != TaskScheduler.Default</span><br><span class="line">    用当前 TaskScheduler 调度 continuation</span><br><span class="line"></span><br><span class="line">否则</span><br><span class="line">    ThreadPool</span><br></pre></td></tr></table></figure><p>默认情况下，普通Console程序，<code>SynchronizationContext.Current</code>是null，且使用TaskScheduler.Default(也就是ThreadPool)作为调度器。</p><p><code>SynchronizationContext</code>是一个线程维度的属性，在ThreadPool中它是null，它用来决定我await一个async函数后，接下来自己的continuation在哪里执行。下面代码展示了如何自定义：</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">using</span> System;</span><br><span class="line"><span class="keyword">using</span> System.Collections.Concurrent;</span><br><span class="line"><span class="keyword">using</span> System.Threading;</span><br><span class="line"><span class="keyword">using</span> System.Threading.Tasks;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">sealed</span> <span class="keyword">class</span> <span class="title">SingleThreadSynchronizationContext</span> : <span class="title">SynchronizationContext</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">private</span> <span class="keyword">readonly</span> BlockingCollection&lt;(SendOrPostCallback Callback, <span class="built_in">object</span>? State)&gt; _queue = <span class="keyword">new</span>();</span><br><span class="line"></span><br><span class="line">    <span class="keyword">private</span> <span class="keyword">readonly</span> <span class="built_in">int</span> _ownerThreadId;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="title">SingleThreadSynchronizationContext</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        _ownerThreadId = Thread.CurrentThread.ManagedThreadId;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">override</span> <span class="keyword">void</span> <span class="title">Post</span>(<span class="params">SendOrPostCallback d, <span class="built_in">object</span>? state</span>)</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="comment">// 异步投递，不阻塞调用方</span></span><br><span class="line">        _queue.Add((d, state));</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">override</span> <span class="keyword">void</span> <span class="title">Send</span>(<span class="params">SendOrPostCallback d, <span class="built_in">object</span>? state</span>)</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="comment">// 如果已经在目标线程上，直接执行</span></span><br><span class="line">        <span class="keyword">if</span> (Thread.CurrentThread.ManagedThreadId == _ownerThreadId)</span><br><span class="line">        &#123;</span><br><span class="line">            d(state);</span><br><span class="line">            <span class="keyword">return</span>;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        <span class="keyword">using</span> <span class="keyword">var</span> done = <span class="keyword">new</span> ManualResetEventSlim(<span class="literal">false</span>);</span><br><span class="line">        Exception? exception = <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line">        Post(s =&gt;</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="keyword">try</span></span><br><span class="line">            &#123;</span><br><span class="line">                d(s);</span><br><span class="line">            &#125;</span><br><span class="line">            <span class="keyword">catch</span> (Exception ex)</span><br><span class="line">            &#123;</span><br><span class="line">                exception = ex;</span><br><span class="line">            &#125;</span><br><span class="line">            <span class="keyword">finally</span></span><br><span class="line">            &#123;</span><br><span class="line">                done.Set();</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;, state);</span><br><span class="line"></span><br><span class="line">        done.Wait();</span><br><span class="line"></span><br><span class="line">        <span class="keyword">if</span> (exception != <span class="literal">null</span>)</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="keyword">throw</span> exception;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">RunOnCurrentThread</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">foreach</span> (<span class="keyword">var</span> workItem <span class="keyword">in</span> _queue.GetConsumingEnumerable())</span><br><span class="line">        &#123;</span><br><span class="line">            workItem.Callback(workItem.State);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">Complete</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        _queue.CompleteAdding();</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title">Program</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="function"><span class="keyword">static</span> <span class="keyword">async</span> Task <span class="title">Main</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">var</span> context = <span class="keyword">new</span> SingleThreadSynchronizationContext();</span><br><span class="line"></span><br><span class="line">        SynchronizationContext.SetSynchronizationContext(context);</span><br><span class="line"></span><br><span class="line">        Console.WriteLine(<span class="string">$&quot;Main start thread: <span class="subst">&#123;Thread.CurrentThread.ManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">var</span> task = DoAsync();</span><br><span class="line"></span><br><span class="line">        task.ContinueWith(_ =&gt; context.Complete());</span><br><span class="line"></span><br><span class="line">        context.RunOnCurrentThread();</span><br><span class="line"></span><br><span class="line">        <span class="keyword">await</span> task;</span><br><span class="line"></span><br><span class="line">        Console.WriteLine(<span class="string">&quot;Program finished.&quot;</span>);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">static</span> <span class="keyword">async</span> Task <span class="title">DoAsync</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        Console.WriteLine(<span class="string">$&quot;Before await thread: <span class="subst">&#123;Thread.CurrentThread.ManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">await</span> Task.Delay(<span class="number">1000</span>);</span><br><span class="line"></span><br><span class="line">        Console.WriteLine(<span class="string">$&quot;After await thread: <span class="subst">&#123;Thread.CurrentThread.ManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// 输出</span></span><br><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment">Main start thread: 1</span></span><br><span class="line"><span class="comment">Before await thread: 1</span></span><br><span class="line"><span class="comment">After await thread: 1</span></span><br><span class="line"><span class="comment">Program finished.</span></span><br><span class="line"><span class="comment">*/</span></span><br></pre></td></tr></table></figure><p>理解上面代码有以下要点：</p><ul><li>SetSynchronizationContext用来设置当前执行线程的SynchronizationContext，简称上下文。</li><li>await task时，如果任务未完成、需要挂起，则会注册continuation，并默认按优先级捕获当前调度环境：<br>优先捕获当前SynchronizationContext；<br>如果没有合适的SynchronizationContext，则可能使用当前非默认TaskScheduler；<br>如果也没有非默认TaskScheduler，则走默认continuation调度逻辑。</li><li>await注册continuation时捕获当前执行点的调度环境，即SynchronizationContext OR TaskScheduler，最后决定task完成时恢复该continuation的位置。<br>如果捕获到了SynchronizationContext，continuation的调度逻辑通常会通过context.Post执行，让自定义SynchronizationContext决定后续代码在哪里运行。<br>如果没有捕获到合适的SynchronizationContext，则可能使用当前非默认TaskScheduler；<br>如果当前TaskScheduler也是默认调度器，则走默认continuation调度逻辑。</li><li>await task时，如果任务已经完成，则此处通常会直接继续执行，不会挂起，也不会把continuation注册到该task上，因此不会经过SynchronizationContext.Post或TaskScheduler调度。</li><li>如果确实想强制把后续代码重新排队调度一次，可以使用 await Task.Yield()。<br>它返回的是一个特殊awaitable，它的awaiter通常会让IsCompleted &#x3D;&#x3D; false，所以它会强制异步让出当前执行流，然后把后续continuation交给当前SynchronizationContext或TaskScheduler调度。<br>但这不是每次await都要配套使用的东西，业务代码中比较罕见，也通常不推荐。  <figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">async</span> Task <span class="title">AwaitWithYieldAsync</span>(<span class="params">Task task</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">await</span> task.ConfigureAwait(<span class="literal">true</span>);</span><br><span class="line">    <span class="keyword">await</span> Task.Yield();</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure></li><li>await task.ConfigureAwait(false) 会禁止 continuation 捕获当前 SynchronizationContext 和当前非默认 TaskScheduler，因此不会强制回到原上下文&#x2F;原调度器执行。</li></ul><p>TaskScheduler决定了一个task continuation被安排到哪里执行，默认的TaskScheduler是<code>TaskScheduler.Default</code>也就是ThreadPool，使用它可以做到这些事情：</p><ul><li>所有任务只在一个线程上执行；</li><li>限制最大并发数；</li><li>给任务排优先级；</li><li>把任务投递到某个专用线程。</li></ul><p>下面是一个自定义TaskScheduler的例子：</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">using</span> System.Collections.Concurrent;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">sealed</span> <span class="keyword">class</span> <span class="title">SingleThreadTaskScheduler</span> : <span class="title">TaskScheduler</span>, <span class="title">IDisposable</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">private</span> <span class="keyword">readonly</span> BlockingCollection&lt;Task&gt; _tasks = <span class="keyword">new</span>();</span><br><span class="line">    <span class="keyword">private</span> <span class="keyword">readonly</span> Thread _thread;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="title">SingleThreadTaskScheduler</span>(<span class="params"><span class="built_in">string</span> name = <span class="string">&quot;SingleThreadTaskScheduler&quot;</span></span>)</span></span><br><span class="line">    &#123;</span><br><span class="line">        _thread = <span class="keyword">new</span> Thread(Run)</span><br><span class="line">        &#123;</span><br><span class="line">            IsBackground = <span class="literal">true</span>,</span><br><span class="line">            Name = name</span><br><span class="line">        &#125;;</span><br><span class="line"></span><br><span class="line">        _thread.Start();</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">Run</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">foreach</span> (<span class="keyword">var</span> task <span class="keyword">in</span> _tasks.GetConsumingEnumerable())</span><br><span class="line">        &#123;</span><br><span class="line">            TryExecuteTask(task);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">protected</span> <span class="keyword">override</span> <span class="keyword">void</span> <span class="title">QueueTask</span>(<span class="params">Task task</span>)</span></span><br><span class="line">    &#123;</span><br><span class="line">        _tasks.Add(task);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">protected</span> <span class="keyword">override</span> <span class="built_in">bool</span> <span class="title">TryExecuteTaskInline</span>(<span class="params">Task task, <span class="built_in">bool</span> taskWasPreviouslyQueued</span>)</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="comment">// 为了保证所有 Task 都在专用线程执行，这里禁止 inline</span></span><br><span class="line">        <span class="keyword">return</span> <span class="literal">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">protected</span> <span class="keyword">override</span> IEnumerable&lt;Task&gt;? GetScheduledTasks()</span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">return</span> _tasks.ToArray();</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">Dispose</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        _tasks.CompleteAdding();</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title">Program</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="function"><span class="keyword">static</span> <span class="keyword">void</span> <span class="title">Main</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">using</span> <span class="keyword">var</span> scheduler = <span class="keyword">new</span> SingleThreadTaskScheduler();</span><br><span class="line"></span><br><span class="line">        <span class="keyword">var</span> factory = <span class="keyword">new</span> TaskFactory(scheduler);</span><br><span class="line"></span><br><span class="line">        Console.WriteLine(<span class="string">$&quot;Main thread: <span class="subst">&#123;Thread.CurrentThread.ManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line"></span><br><span class="line">        Task task1 = factory.StartNew(() =&gt;</span><br><span class="line">        &#123;</span><br><span class="line">            Console.WriteLine(<span class="string">$&quot;Task1 thread: <span class="subst">&#123;Thread.CurrentThread.ManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line">            Console.WriteLine(<span class="string">$&quot;Task1 scheduler is custom: <span class="subst">&#123;TaskScheduler.Current == scheduler&#125;</span>&quot;</span>);</span><br><span class="line">        &#125;);</span><br><span class="line"></span><br><span class="line">        Task&lt;<span class="built_in">int</span>&gt; task2 = factory.StartNew(() =&gt;</span><br><span class="line">        &#123;</span><br><span class="line">            Console.WriteLine(<span class="string">$&quot;Task2 thread: <span class="subst">&#123;Thread.CurrentThread.ManagedThreadId&#125;</span>&quot;</span>);</span><br><span class="line">            Console.WriteLine(<span class="string">$&quot;Task2 scheduler is custom: <span class="subst">&#123;TaskScheduler.Current == scheduler&#125;</span>&quot;</span>);</span><br><span class="line">            <span class="keyword">return</span> <span class="number">123</span>;</span><br><span class="line">        &#125;);</span><br><span class="line"></span><br><span class="line">        Task.WaitAll(task1, task2);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>可以使用TaskFactory自定义一个新的环境，设置这个环境的TaskScheduler。值得一提的是如果当前同时设置了上下文和调度器，调度器其实是没有用的，形同虚设。</p><p>这里解答下为什么没有指定异步参数的TaskCompletionSource.SetResult为什么仅仅是可能在原地跑(更确切的说是内联执行)，原因就是Task完成时，会看被注册在自己身上的continuation的运行环境(上下文或者调度器)，如果它和SetResult处的运行环境相同，就没必要丢队列了，自己执行也OK。出于性能的目的，才有了内联执行的概念。</p><h1 id="UI协程"><a href="#UI协程" class="headerlink" title="UI协程"></a>UI协程</h1><p>在很多游戏框架里面，允许使用协程让代码更好读，避免回调地狱。下面是一段回调地狱代码：</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">Button_Click</span>(<span class="params"><span class="built_in">object</span> sender, EventArgs e</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    button.Enabled = <span class="literal">false</span>;</span><br><span class="line">    textBox.Text = <span class="string">&quot;Loading...&quot;</span>;</span><br><span class="line"></span><br><span class="line">    DoHttpGetAsync(<span class="string">&quot;https://example.com/user&quot;</span>, userResult =&gt;</span><br><span class="line">    &#123;</span><br><span class="line">        <span class="comment">// 回到 UI 线程</span></span><br><span class="line">        <span class="keyword">this</span>.BeginInvoke(<span class="keyword">new</span> Action(() =&gt;</span><br><span class="line">        &#123;</span><br><span class="line">            textBox.Text = <span class="string">&quot;User loaded: &quot;</span> + userResult;</span><br><span class="line"></span><br><span class="line">            DoHttpGetAsync(<span class="string">&quot;https://example.com/orders?user=&quot;</span> + userResult, ordersResult =&gt;</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="comment">// 再回到 UI 线程</span></span><br><span class="line">                <span class="keyword">this</span>.BeginInvoke(<span class="keyword">new</span> Action(() =&gt;</span><br><span class="line">                &#123;</span><br><span class="line">                    textBox.Text += Environment.NewLine + <span class="string">&quot;Orders loaded: &quot;</span> + ordersResult;</span><br><span class="line"></span><br><span class="line">                    DoHttpGetAsync(<span class="string">&quot;https://example.com/detail?order=&quot;</span> + ordersResult, detailResult =&gt;</span><br><span class="line">                    &#123;</span><br><span class="line">                        <span class="comment">// 再回到 UI 线程</span></span><br><span class="line">                        <span class="keyword">this</span>.BeginInvoke(<span class="keyword">new</span> Action(() =&gt;</span><br><span class="line">                        &#123;</span><br><span class="line">                            textBox.Text += Environment.NewLine + <span class="string">&quot;Detail loaded: &quot;</span> + detailResult;</span><br><span class="line"></span><br><span class="line">                            DoHttpPostAsync(<span class="string">&quot;https://example.com/log&quot;</span>, detailResult, logResult =&gt;</span><br><span class="line">                            &#123;</span><br><span class="line">                                <span class="comment">// 再回到 UI 线程</span></span><br><span class="line">                                <span class="keyword">this</span>.BeginInvoke(<span class="keyword">new</span> Action(() =&gt;</span><br><span class="line">                                &#123;</span><br><span class="line">                                    textBox.Text += Environment.NewLine + <span class="string">&quot;Log saved: &quot;</span> + logResult;</span><br><span class="line">                                    button.Enabled = <span class="literal">true</span>;</span><br><span class="line">                                &#125;));</span><br><span class="line">                            &#125;,</span><br><span class="line">                            ex =&gt;</span><br><span class="line">                            &#123;</span><br><span class="line">                                <span class="keyword">this</span>.BeginInvoke(<span class="keyword">new</span> Action(() =&gt;</span><br><span class="line">                                &#123;</span><br><span class="line">                                    MessageBox.Show(<span class="string">&quot;Save log failed: &quot;</span> + ex.Message);</span><br><span class="line">                                    button.Enabled = <span class="literal">true</span>;</span><br><span class="line">                                &#125;));</span><br><span class="line">                            &#125;);</span><br><span class="line">                        &#125;));</span><br><span class="line">                    &#125;,</span><br><span class="line">                    ex =&gt;</span><br><span class="line">                    &#123;</span><br><span class="line">                        <span class="keyword">this</span>.BeginInvoke(<span class="keyword">new</span> Action(() =&gt;</span><br><span class="line">                        &#123;</span><br><span class="line">                            MessageBox.Show(<span class="string">&quot;Load detail failed: &quot;</span> + ex.Message);</span><br><span class="line">                            button.Enabled = <span class="literal">true</span>;</span><br><span class="line">                        &#125;));</span><br><span class="line">                    &#125;);</span><br><span class="line">                &#125;));</span><br><span class="line">            &#125;,</span><br><span class="line">            ex =&gt;</span><br><span class="line">            &#123;</span><br><span class="line">                <span class="keyword">this</span>.BeginInvoke(<span class="keyword">new</span> Action(() =&gt;</span><br><span class="line">                &#123;</span><br><span class="line">                    MessageBox.Show(<span class="string">&quot;Load orders failed: &quot;</span> + ex.Message);</span><br><span class="line">                    button.Enabled = <span class="literal">true</span>;</span><br><span class="line">                &#125;));</span><br><span class="line">            &#125;);</span><br><span class="line">        &#125;));</span><br><span class="line">    &#125;,</span><br><span class="line">    ex =&gt;</span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">this</span>.BeginInvoke(<span class="keyword">new</span> Action(() =&gt;</span><br><span class="line">        &#123;</span><br><span class="line">            MessageBox.Show(<span class="string">&quot;Load user failed: &quot;</span> + ex.Message);</span><br><span class="line">            button.Enabled = <span class="literal">true</span>;</span><br><span class="line">        &#125;));</span><br><span class="line">    &#125;);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>但是如果有了协程，生活就更简单了，整个流程看起来就是<strong>同步</strong>的：</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">async</span> <span class="keyword">void</span> <span class="title">Button_Click</span>(<span class="params"><span class="built_in">object</span> sender, EventArgs e</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    button.Enabled = <span class="literal">false</span>;</span><br><span class="line">    textBox.Text = <span class="string">&quot;Loading...&quot;</span>;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">try</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="built_in">string</span> user = <span class="keyword">await</span> DoHttpGetAsync(<span class="string">&quot;https://example.com/user&quot;</span>);</span><br><span class="line">        textBox.Text = <span class="string">&quot;User loaded: &quot;</span> + user;</span><br><span class="line"></span><br><span class="line">        <span class="built_in">string</span> orders = <span class="keyword">await</span> DoHttpGetAsync(<span class="string">&quot;https://example.com/orders?user=&quot;</span> + user);</span><br><span class="line">        textBox.Text += Environment.NewLine + <span class="string">&quot;Orders loaded: &quot;</span> + orders;</span><br><span class="line"></span><br><span class="line">        <span class="built_in">string</span> detail = <span class="keyword">await</span> DoHttpGetAsync(<span class="string">&quot;https://example.com/detail?order=&quot;</span> + orders);</span><br><span class="line">        textBox.Text += Environment.NewLine + <span class="string">&quot;Detail loaded: &quot;</span> + detail;</span><br><span class="line"></span><br><span class="line">        <span class="built_in">string</span> log = <span class="keyword">await</span> DoHttpPostAsync(<span class="string">&quot;https://example.com/log&quot;</span>, detail);</span><br><span class="line">        textBox.Text += Environment.NewLine + <span class="string">&quot;Log saved: &quot;</span> + log;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">catch</span> (Exception ex)</span><br><span class="line">    &#123;</span><br><span class="line">        MessageBox.Show(ex.Message);</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">finally</span></span><br><span class="line">    &#123;</span><br><span class="line">        button.Enabled = <span class="literal">true</span>;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>上面的做法避免的回调地狱，看起来是同步代码，实际上底层是异步的。</p><h1 id="UI协程的原理"><a href="#UI协程的原理" class="headerlink" title="UI协程的原理"></a>UI协程的原理</h1><p>一个典型游戏程序通常有一个主线程上的帧循环，它被称为帧循环即frame loop。</p><p>这个循环不断poll外部事件(比如定时器，窗口系统事件，输入事件…)，然后根据上一帧到当前帧的时间差 deltaTime 更新游戏世界，最后渲染当前帧并提交给显示系统。</p><p>下面是伪代码，可以看个意思：</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> gameContext = <span class="keyword">new</span> GameSynchronizationContext();</span><br><span class="line">SynchronizationContext.SetSynchronizationContext(gameContext);</span><br><span class="line"></span><br><span class="line"><span class="keyword">while</span> (running)</span><br><span class="line">&#123;</span><br><span class="line">    <span class="comment">// 尽早处理窗口消息，避免系统认为程序无响应</span></span><br><span class="line">    PollEvents();</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 处理投递回主线程的回调：</span></span><br><span class="line">    <span class="comment">// - async/await continuation</span></span><br><span class="line">    <span class="comment">// - RunInLoop/Post 回调</span></span><br><span class="line">    <span class="comment">// - 其他用户态主线程任务</span></span><br><span class="line">    gameContext.ExecutePendingCallbacks();</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 计算时间</span></span><br><span class="line">    <span class="built_in">double</span> dt = timer.GetDeltaTime();</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 更新世界</span></span><br><span class="line">    Update(dt);</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 尽量靠近渲染前再采样关键输入</span></span><br><span class="line">    PollInputLate();</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 渲染当前帧</span></span><br><span class="line">    Render();</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 提交帧，可能阻塞在 VSync</span></span><br><span class="line">    Present();</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">sealed</span> <span class="keyword">class</span> <span class="title">GameSynchronizationContext</span> : <span class="title">SynchronizationContext</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">private</span> <span class="keyword">readonly</span> ConcurrentQueue&lt;(SendOrPostCallback Callback, <span class="built_in">object</span>? State)&gt; _queue = <span class="keyword">new</span>();</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">override</span> <span class="keyword">void</span> <span class="title">Post</span>(<span class="params">SendOrPostCallback d, <span class="built_in">object</span>? state</span>)</span></span><br><span class="line">    &#123;</span><br><span class="line">        _queue.Enqueue((d, state));</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">ExecutePendingCallbacks</span>()</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">while</span> (_queue.TryDequeue(<span class="keyword">out</span> <span class="keyword">var</span> item))</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="keyword">try</span></span><br><span class="line">            &#123;</span><br><span class="line">                item.Callback(item.State);</span><br><span class="line">            &#125;</span><br><span class="line">            <span class="keyword">catch</span> (Exception ex)</span><br><span class="line">            &#123;</span><br><span class="line">                Console.WriteLine(<span class="string">&quot;Async continuation exception: &quot;</span> + ex);</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">void</span> <span class="title">PollEvents</span>()</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="comment">// 操作系统来的事件：键盘/鼠标/手柄输入，窗口状态(resize/关闭/最小化/移动/失去获得焦点/DPI变化)，文件拖拽，窗口重绘</span></span><br><span class="line">    PollWindowEvents();</span><br><span class="line">    <span class="comment">// linux有timerfd，会比较方便</span></span><br><span class="line">    PollTimerEvents();</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>关于窗口重绘事件：游戏是主动去绘制然后调用GPU进行Swap Buffer的，并不是窗口重绘事件来了才进行绘制并进行Swap，因此这个窗口重绘事件其实在游戏中不重要。</p><p>像记事本这类传统GUI程序通常是事件驱动的，不会像游戏一样每帧主动重绘。它们主要在窗口内容失效、窗口大小变化、被遮挡后重新显示、文本变化等情况下，由系统发送重绘&#x2F;绘制事件。程序收到绘制事件后，需要在系统指定的绘制上下文中把失效区域重新画出来，并在绘制结束后通知系统该区域已经有效。</p><h1 id="network-async底层"><a href="#network-async底层" class="headerlink" title="network async底层"></a>network async底层</h1><p>现在有的同学肯定很好奇，这么好用的network async库底层是怎么实现的呢，这里我给出一个linux的实现方法：</p><figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br></pre></td><td class="code"><pre><span class="line">await stream.ReadAsync(buffer)</span><br><span class="line">        ↓</span><br><span class="line">NetworkStream.ReadAsync</span><br><span class="line">        ↓</span><br><span class="line">Socket.ReceiveAsync / ReceiveAsync(SocketAsyncEventArgs)</span><br><span class="line">        ↓</span><br><span class="line">进入该 socket 对应的 SocketAsyncContext</span><br><span class="line">        ↓</span><br><span class="line">构造或复用一个 receive/read operation</span><br><span class="line">        ↓</span><br><span class="line">进入读方向 operation queue 的调度逻辑</span><br><span class="line">        ↓</span><br><span class="line">如果前面没有更早的 pending read，并且 non-blocking recv 立即读到数据：</span><br><span class="line">    同步完成本次 operation</span><br><span class="line">    返回已完成的 Task/ValueTask</span><br><span class="line">        ↓</span><br><span class="line">如果当前 fd 暂时不可读，例如 recv 返回 EAGAIN/EWOULDBLOCK，</span><br><span class="line">或前面已经有更早的 pending read：</span><br><span class="line">    把本次 operation 留在 read operation queue 中</span><br><span class="line">    返回尚未完成的 Task/ValueTask</span><br><span class="line">        ↓</span><br><span class="line">await 发现 Task/ValueTask 尚未完成</span><br><span class="line">        ↓</span><br><span class="line">当前 async 方法状态机挂起，continuation 注册到这个异步结果上</span><br><span class="line">        ↓</span><br><span class="line">SocketAsyncEngine 通过 epoll 等待该 fd 的 readable 事件</span><br><span class="line">        ↓</span><br><span class="line">.NET Sockets 事件线程收到 epoll 事件</span><br><span class="line">        ↓</span><br><span class="line">SocketAsyncEngine 根据事件里的索引/数据找到对应 SocketAsyncContext</span><br><span class="line">        ↓</span><br><span class="line">默认把对应 I/O 事件投递给 ThreadPool 处理</span><br><span class="line">        ↓</span><br><span class="line">ThreadPool 线程推进 SocketAsyncContext 的读队列</span><br><span class="line">        ↓</span><br><span class="line">按顺序取出 pending read/receive operation</span><br><span class="line">        ↓</span><br><span class="line">再次尝试 non-blocking recv</span><br><span class="line">        ↓</span><br><span class="line">如果成功读到数据或读到 EOF：</span><br><span class="line">    完成该 operation</span><br><span class="line">    设置读取字节数 / socket 状态</span><br><span class="line">    完成对应 Task/ValueTask</span><br><span class="line">        ↓</span><br><span class="line">如果仍然返回 EAGAIN/EWOULDBLOCK：</span><br><span class="line">    operation 继续保持 pending</span><br><span class="line">    等待后续 readable 事件</span><br><span class="line">        ↓</span><br><span class="line">Task/ValueTask 完成后</span><br><span class="line">        ↓</span><br><span class="line">await continuation 按捕获到的 SynchronizationContext / TaskScheduler / ThreadPool 规则恢复执行</span><br></pre></td></tr></table></figure><p>每个socket都有自己的<code>SocketAsyncContext</code>，其中包含<code>read operation queue</code>和<code>write operation queue</code>两个操作队列。read队列里面可以存储类读事件比如<code>Receive</code>、<code>Accept</code>。write队列里面存储的事件比如<code>Write/Send/SendFile/Connect</code>。需要对读写做排队是因为读&#x2F;写讲究一个顺序问题，需要按先后顺序进行read操作，至少不能让后ReadAsync的人拿到靠前的数据。</p><p><em>SocketAsyncEngine 根据事件里的索引&#x2F;数据找到对应 SocketAsyncContext</em>，这里的索引指的是这样的数据结构：</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> SocketAsyncContext?[] s_registeredContexts = [];</span><br><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">readonly</span> Queue&lt;<span class="built_in">int</span>&gt; s_registeredContextsFreeList = [];</span><br></pre></td></tr></table></figure><p><code>epoll_event.data</code>里面有一个index，可以通过<code>s_registeredContexts[index]</code>拿到真正的SocketAsyncContext。</p>]]>
    </content>
    <id>https://markity.github.io/csharp-concurrent-model/</id>
    <link href="https://markity.github.io/csharp-concurrent-model/"/>
    <published>2026-05-28T04:10:00.000Z</published>
    <summary>
      <![CDATA[<p>不同于Go的<code>goroutine</code>和Java 21的<code>virtual threads</code>这类“有栈协程”模型，C#和Python更常使用<code>async/await</code>的“无栈协程”模型来处理并发。本文探索C#的<code>async/await</code>并发模型，以及它与Go有栈协程的差异。</p>]]>
    </summary>
    <title>探索C# async/await无栈协程并发模型</title>
    <updated>2026-05-28T04:10:00.000Z</updated>
  </entry>
  <entry>
    <author>
      <name>Markity</name>
    </author>
    <content>
      <![CDATA[<p>Hello！我目前是一个后端程序员，目前正在学习图形学，Linux，以及桌面软件等生态，梦想以后成为独立游戏开发者。目前我将在这里记录我的所有成长过程！</p><span id="more"></span><h1 id="Favorite-Things"><a href="#Favorite-Things" class="headerlink" title="Favorite Things"></a>Favorite Things</h1><ul><li>CS2</li><li>瓦罗兰特</li><li>我的世界</li><li>动漫</li></ul><h1 id="熟悉的技术"><a href="#熟悉的技术" class="headerlink" title="熟悉的技术"></a>熟悉的技术</h1><ul><li>Golang及后台开发技术</li><li>Java及后台开发技术</li><li>Epoll&#x2F;Reactor及网络开发技术</li><li>消息&#x2F;IM技术</li><li>社区评论系统</li><li>Raft协议</li></ul><h1 id="正在学习"><a href="#正在学习" class="headerlink" title="正在学习"></a>正在学习</h1><ul><li>Godot</li><li>Games系列课程</li><li>Opengl</li><li>Wayland Client编程</li></ul><h1 id="推荐的书"><a href="#推荐的书" class="headerlink" title="推荐的书"></a>推荐的书</h1><ul><li>C程序设计语言，作者K&amp;R（C语言圣经，黑皮书）</li><li>Linux高性能服务器编程，作者陈硕（除了事件驱动的网络模型，还讲解内存管理相关的最佳实践）</li><li>Go程序设计语言（Go语言圣经，黑皮书）</li></ul>]]>
    </content>
    <id>https://markity.github.io/about/</id>
    <link href="https://markity.github.io/about/"/>
    <published>2026-04-08T05:34:00.000Z</published>
    <summary>
      <![CDATA[<p>Hello！我目前是一个后端程序员，目前正在学习图形学，Linux，以及桌面软件等生态，梦想以后成为独立游戏开发者。目前我将在这里记录我的所有成长过程！</p>]]>
    </summary>
    <title>关于博客</title>
    <updated>2026-04-08T05:34:00.000Z</updated>
  </entry>
</feed>
