【.NET】WebAPi之?dāng)帱c續(xù)傳下載（中）

_明心見性_ 2019-07-19

展開全文

前言

前情回顧：上一篇我們遺留了兩個問題，一個是未完全實現(xiàn)斷點續(xù)傳，另外則是在響應(yīng)時是返回StreamContent還是PushStreamContent呢,？這一節(jié)我們重點來解決這兩個問題,，同時就在此過程中需要注意的地方一并指出,，若有錯誤之處,，請指出,。

StreamContent compare to PushStreamContent

我們來看看StreamContent代碼,，如下：

public class StreamContent : HttpContent
{
    // Fields
    private int bufferSize;
    private Stream content;
    private bool contentConsumed;
    private const int defaultBufferSize = 0x1000;
    private long start;
    // Methods
    public StreamContent(Stream content);
]
    public StreamContent(Stream content, int bufferSize);
    protected override Task<Stream> CreateContentReadStreamAsync();
    protected override void Dispose(bool disposing);
    private void PrepareContent();
    protected override Task SerializeToStreamAsync(Stream stream, TransportContext context);
    protected internal override bool TryComputeLength(out long length);
    // Nested Types
    private class ReadOnlyStream : DelegatingStream
    {......}
}

似乎沒有什么可看的,，但是有一句話我們需要注意，如下：

 private const int defaultBufferSize = 0x1000;

在StreamContent的第二個構(gòu)造函數(shù)為

 public StreamContent(Stream content, int bufferSize);

上述給定的默認一次性輸入到緩沖區(qū)大小為4k,，這對我們有何意義呢,？當(dāng)我們寫入到響應(yīng)中時，一般我們直接利用的是第一個構(gòu)造函數(shù),，如下：

  var response = new HttpResponseMessage();
  response.Content = new StreamContent(fileStream);

到這里我們明白了這么做是有問題的,，當(dāng)下載時默認讀取的是4k，如果文件比較大下載的時間則有延長,，所以我們在返回時一定要給定緩沖大小,，那么給定多少呢？為達到更好的性能最多是80k，如下：

 private const int BufferSize = 80 * 1024;
 response.Content = new StreamContent(fileStream, BufferSize);

此時下載的速度則有很大的改善,，有人就說了為何是80k呢,？這個問題我也不知道，老外驗證過的,，這是鏈接【.NET Asynchronous stream read/write】,。

好了說完StreamContent，接下來我們來看看PushStreamContent,，從字面意思來為推送流內(nèi)容,，難道是充分利用了緩沖區(qū)嗎，猜測可以有,，就怕沒有任何想法，我們用源碼來證明看看,。

我們只需看看WebHost模式下對于緩沖策略是怎么選擇的,，我們看看此類 WebHostBufferPolicySelector 實現(xiàn)，代碼如下：

     /// <summary>
    /// Provides an implementation of <see cref="IHostBufferPolicySelector"/> suited for use
    /// in an ASP.NET environment which provides direct support for input and output buffering.
    /// </summary>
    public class WebHostBufferPolicySelector : IHostBufferPolicySelector
    {
        ....../// <summary>
        /// Determines whether the host should buffer the <see cref="HttpResponseMessage"/> entity body.
        /// </summary>
        /// <param name="response">The <see cref="HttpResponseMessage"/>response for which to determine
        /// whether host output buffering should be used for the response entity body.</param>
        /// <returns><c>true</c> if buffering should be used; otherwise a streamed response should be used.</returns>
        public virtual bool UseBufferedOutputStream(HttpResponseMessage response)
        {
            if (response == null)
            {
                throw Error.ArgumentNull("response");
            }
            // Any HttpContent that knows its length is presumably already buffered internally.
            HttpContent content = response.Content;
            if (content != null)
            {
                long? contentLength = content.Headers.ContentLength;
                if (contentLength.HasValue && contentLength.Value >= 0)
                {
                    return false;
                }
                // Content length is null or -1 (meaning not known).  
                // Buffer any HttpContent except StreamContent and PushStreamContent
                return !(content is StreamContent || content is PushStreamContent);
            }
            return false;
        }
    }

從上述如下一句可以很明顯的知道：

 return !(content is StreamContent || content is PushStreamContent);

除了StreamContent和PushStreamContent的HttpContent之外,，其余都進行緩沖,，所以二者的區(qū)別不在于緩沖，那到底是什么呢,？好了我們還未查看PushStreamContent的源碼,，我們繼續(xù)往下走，查看其源代碼如下,，我們僅僅只看關(guān)于這個類的描述以及第一個構(gòu)造函數(shù)即可,，如下：

  /// <summary>
    /// Provides an <see cref="HttpContent"/> implementation that exposes an output <see cref="Stream"/>
    /// which can be written to directly. The ability to push data to the output stream differs from the 
    /// <see cref="StreamContent"/> where data is pulled and not pushed.
    /// </summary>
    public class PushStreamContent : HttpContent
    {
        private readonly Func<Stream, HttpContent, TransportContext, Task> _onStreamAvailable;
        /// <summary>
        /// Initializes a new instance of the <see cref="PushStreamContent"/> class. The
        /// <paramref name="onStreamAvailable"/> action is called when an output stream
        /// has become available allowing the action to write to it directly. When the 
        /// stream is closed, it will signal to the content that is has completed and the 
        /// HTTP request or response will be completed.
        /// </summary>
        /// <param name="onStreamAvailable">The action to call when an output stream is available.</param>
        public PushStreamContent(Action<Stream, HttpContent, TransportContext> onStreamAvailable)
            : this(Taskify(onStreamAvailable), (MediaTypeHeaderValue)null)
        {
        }
        ......
   }

對于此類的描述大意是：PushStreamContent與StreamContent的不同在于，PushStreamContent在于將數(shù)據(jù)push【推送】到輸出流中,，而StreamContent則是將數(shù)據(jù)從流中【拉取】,。

貌似有點晦澀，我們來舉個例子,，在webapi中我們常常這樣做,，讀取文件流并返回到響應(yīng)流中，若是StreamContent,，我們會如下這樣做：

response.Content = new StreamContent(File.OpenRead(filePath));

上面的釋義我用大括號著重括起,，StreamContent著重于【拉取】，當(dāng)響應(yīng)時此時將從文件流寫到輸出流,，通俗一點說則是我們需要從文件流中去獲取數(shù)據(jù)并寫入到輸出流中,。我們再來看看PushStreamContent的用法，如下：

XDocument xDoc = XDocument.Load("cnblogs_backup.xml", LoadOptions.None);
PushStreamContent xDocContent = new PushStreamContent(
(stream, content, context) =>
{
     xDoc.Save(stream);
     stream.Close();
},
"application/xml");

PushStreamContent著重于【推送】,，當(dāng)我們加載xml文件時,，當(dāng)我們一旦進行保存時此時則會將數(shù)據(jù)推送到輸出流中。

二者區(qū)別在于：StreamContent從流中【拉取】數(shù)據(jù)，而PushStreamContent則是將數(shù)據(jù)【推送】到流中,。

那么此二者應(yīng)用的場景是什么呢,？

（1）對于下載文件我們則可以通過StreamContent來實現(xiàn)直接從流中拉取，若下載視頻流此時則應(yīng)該利用PushStreamContent來實現(xiàn),，因為未知服務(wù)器視頻資源的長度,，此視頻資源來源于別的地方。

（2）數(shù)據(jù)量巨大,，發(fā)送請求到webapi時利用PushStreamContent,。

當(dāng)發(fā)送請求時，常常序列化數(shù)據(jù)并請求webapi,，我們可能這樣做：

    var client = new HttpClient();
    string json = JsonConvert.SerializeObject(data);
    var response = await client.PostAsync(uri, new StringContent(json));

當(dāng)數(shù)據(jù)量比較小時沒問題,，若數(shù)據(jù)比較大時進行序列化此時則將序列化的字符串加載到內(nèi)存中，鑒于此這么做不可行,，此時我們應(yīng)該利用PushStreamContent來實現(xiàn),。

    var client = new HttpClient();
    var content = new PushStreamContent((stream, httpContent, transportContext) =>
    {
        var serializer = new JsonSerializer();
        using (var writer = new StreamWriter(stream))
        {
            serializer.Serialize(writer, data);
        }
    });
    var response = await client.PostAsync(uri, content);

為什么要這樣做呢？我們再來看看源碼,，里面存在這樣一個方法,。

  protected override Task SerializeToStreamAsync(Stream stream, TransportContext context);

其內(nèi)部實現(xiàn)利用異步狀態(tài)機實現(xiàn)，所以當(dāng)數(shù)據(jù)量巨大時利用PushStreamContent來返回將會有很大的改善,，至此,，關(guān)于二者的區(qū)別以及常見的應(yīng)用場景已經(jīng)敘述完畢，接下來我們繼續(xù)斷點續(xù)傳問題,。

斷點續(xù)傳改進

上一篇我們講過獲取Range屬性中的集合通過如下：

request.Headers.Range

我們只取該集合中的第一個范圍元素,，通過如下

 RangeItemHeaderValue range = rangeHeader.Ranges.First();

此時我們忽略了返回的該范圍對象中有當(dāng)前下載的進度

range.From.HasValue  
range.To.HasValue

我們獲取二者的值然后進行重寫Stream實時讀取剩余部分，下面我們一步一步來看,。

定義文件操作接口

    public interface IFileProvider
    {
        bool Exists(string name);
        FileStream Open(string name);
        long GetLength(string name);
    }

實現(xiàn)該操作文件接口

    public class FileProvider : IFileProvider
    {
        private readonly string _filesDirectory;
        private const string AppSettingsKey = "DownloadDir";
        public FileProvider()
        {
            var fileLocation = ConfigurationManager.AppSettings[AppSettingsKey];
            if (!String.IsNullOrWhiteSpace(fileLocation))
            {
                _filesDirectory = fileLocation;
            }
        }
        /// <summary>
        /// 判斷文件是否存在
        /// </summary>
        /// <param name="name"></param>
        /// <returns></returns>
        public bool Exists(string name)
        {
            string file = Directory.GetFiles(_filesDirectory, name, SearchOption.TopDirectoryOnly)
                    .FirstOrDefault();
            return true;
        }
        /// <summary>
        /// 打開文件
        /// </summary>
        /// <param name="name"></param>
        /// <returns></returns>
        public FileStream Open(string name)
        {
            var fullFilePath = Path.Combine(_filesDirectory, name);
            return File.Open(fullFilePath,
                FileMode.Open, FileAccess.Read, FileShare.Read);
        }
        /// <summary>
        /// 獲取文件長度
        /// </summary>
        /// <param name="name"></param>
        /// <returns></returns>
        public long GetLength(string name)
        {
            var fullFilePath = Path.Combine(_filesDirectory, name);
            return new FileInfo(fullFilePath).Length;
        }
    }

獲取范圍對象中的值進行賦值給封裝的對象

    public class FileInfo
    {
        public long From;
        public long To;
        public bool IsPartial;
        public long Length;
    }

下載控制器,，對文件操作進行初始化

    public class FileDownloadController : ApiController
    {
        private const int BufferSize = 80 * 1024;
        private const string MimeType = "application/octet-stream";
        public IFileProvider FileProvider { get; set; }
        public FileDownloadController()
        {
            FileProvider = new FileProvider();
        }
        ......
    }

接下來則是文件下載的邏輯，首先判斷請求文件是否存在,，然后獲取文件的長度

            if (!FileProvider.Exists(fileName))
            {
                throw new HttpResponseException(HttpStatusCode.NotFound);
            }
            long fileLength = FileProvider.GetLength(fileName);

將請求中的范圍對象From和To的值并判斷當(dāng)前已經(jīng)下載進度以及剩余進度

        private FileInfo GetFileInfoFromRequest(HttpRequestMessage request, long entityLength)
        {
            var fileInfo = new FileInfo
            {
                From = 0,
                To = entityLength - 1,
                IsPartial = false,
                Length = entityLength
            };
            var rangeHeader = request.Headers.Range;
            if (rangeHeader != null && rangeHeader.Ranges.Count != 0)
            {
                if (rangeHeader.Ranges.Count > 1)
                {
                    throw new HttpResponseException(HttpStatusCode.RequestedRangeNotSatisfiable);
                }
                RangeItemHeaderValue range = rangeHeader.Ranges.First();
                if (range.From.HasValue && range.From < 0 || range.To.HasValue && range.To > entityLength - 1)
                {
                    throw new HttpResponseException(HttpStatusCode.RequestedRangeNotSatisfiable);
                }
                fileInfo.From = range.From ?? 0;
                fileInfo.To = range.To ?? entityLength - 1;
                fileInfo.IsPartial = true;
                fileInfo.Length = entityLength;
                if (range.From.HasValue && range.To.HasValue)
                {
                    fileInfo.Length = range.To.Value - range.From.Value + 1;
                }
                else if (range.From.HasValue)
                {
                    fileInfo.Length = entityLength - range.From.Value + 1;
                }
                else if (range.To.HasValue)
                {
                    fileInfo.Length = range.To.Value + 1;
                }
            }
            return fileInfo;
        }

在響應(yīng)頭信息中的對象ContentRangeHeaderValue設(shè)置當(dāng)前下載進度以及其他響應(yīng)信息

         private void SetResponseHeaders(HttpResponseMessage response, FileInfo fileInfo,
                                      long fileLength, string fileName)
        {
            response.Headers.AcceptRanges.Add("bytes");
            response.StatusCode = fileInfo.IsPartial ? HttpStatusCode.PartialContent
                                      : HttpStatusCode.OK;
            response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
            response.Content.Headers.ContentDisposition.FileName = fileName;
            response.Content.Headers.ContentType = new MediaTypeHeaderValue(MimeType);
            response.Content.Headers.ContentLength = fileInfo.Length;
            if (fileInfo.IsPartial)
            {
                response.Content.Headers.ContentRange
                    = new ContentRangeHeaderValue(fileInfo.From, fileInfo.To, fileLength);
            }
        }

最重要的一步則是將FileInfo對象的值傳遞給我們自定義實現(xiàn)的流監(jiān)控當(dāng)前下載進度,。

    public class PartialContentFileStream : Stream
    {
        private readonly long _start;
        private readonly long _end;
        private long _position;
        private FileStream _fileStream;
        public PartialContentFileStream(FileStream fileStream, long start, long end)
        {
            _start = start;
            _position = start;
            _end = end;
            _fileStream = fileStream;
            if (start > 0)
            {
                _fileStream.Seek(start, SeekOrigin.Begin);
            }
        }
        /// <summary>
        /// 將緩沖區(qū)數(shù)據(jù)寫到文件
        /// </summary>
        public override void Flush()
        {
            _fileStream.Flush();
        }
        /// <summary>
        /// 設(shè)置當(dāng)前下載位置
        /// </summary>
        /// <param name="offset"></param>
        /// <param name="origin"></param>
        /// <returns></returns>
        public override long Seek(long offset, SeekOrigin origin)
        {
            if (origin == SeekOrigin.Begin)
            {
                _position = _start + offset;
                return _fileStream.Seek(_start + offset, origin);
            }
            else if (origin == SeekOrigin.Current)
            {
                _position += offset;
                return _fileStream.Seek(_position + offset, origin);
            }
            else
            {
                throw new NotImplementedException("SeekOrigin.End未實現(xiàn)");
            }
        }
        /// <summary>
        /// 依據(jù)偏離位置讀取
        /// </summary>
        /// <param name="buffer"></param>
        /// <param name="offset"></param>
        /// <param name="count"></param>
        /// <returns></returns>
        public override int Read(byte[] buffer, int offset, int count)
        {
            int byteCountToRead = count;
            if (_position + count > _end)
            {
                byteCountToRead = (int)(_end - _position) + 1;
            }
            var result = _fileStream.Read(buffer, offset, byteCountToRead);
            _position += byteCountToRead;
            return result;
        }
        public override IAsyncResult BeginRead(byte[] buffer, int offset, int count, AsyncCallback callback, object state)
        {
            int byteCountToRead = count;
            if (_position + count > _end)
            {
                byteCountToRead = (int)(_end - _position);
            }
            var result = _fileStream.BeginRead(buffer, offset,
                                               count, (s) =>
                                               {
                                                   _position += byteCountToRead;
                                                   callback(s);
                                               }, state);
            return result;
        }
        ......
    }

更新上述下載的完整邏輯

        public HttpResponseMessage GetFile(string fileName)
        {
            fileName = "HBuilder.windows.5.2.6.zip";
            if (!FileProvider.Exists(fileName))
            {
                throw new HttpResponseException(HttpStatusCode.NotFound);
            }
            long fileLength = FileProvider.GetLength(fileName);
            var fileInfo = GetFileInfoFromRequest(this.Request, fileLength);
            var stream = new PartialContentFileStream(FileProvider.Open(fileName),
                                                 fileInfo.From, fileInfo.To);
            var response = new HttpResponseMessage();
            response.Content = new StreamContent(stream, BufferSize);
            SetResponseHeaders(response, fileInfo, fileLength, fileName);
            return response;
        }

下面我們來看看演示結(jié)果：

好了，到了這里我們也得到了我們想要的結(jié)果,。

總結(jié)

本節(jié)我們將上節(jié)遺留的問題一一進行比較詳細的敘述并最終解決,，是不是就這么完全結(jié)束了呢？那本節(jié)定義為中篇豈不是不對頭了,，本節(jié)是在web端進行下載,，下節(jié)我們利用webclient來進行斷點續(xù)傳。想了想無論是mvc上傳下載,，還是利用webapi來上傳下載又或者是將mvc和webapi結(jié)合來上傳下載基本都已經(jīng)囊括,，這都算是在項目中比較常用的吧,，所以也就花了很多時間去研究。對于webapi的斷點續(xù)傳關(guān)鍵它本身就提供了比較多的api來給我們調(diào)用,，所以還是很不錯,，webapi一個很輕量的服務(wù)框架，你值得擁有see u,，反正周末,，喲，不早了,，休息休息,。