Django streaming response Chinese CSV example

Time:2021-10-25

In Django, streaming responseStreamingHttpResponseIt is a good thing to generate a large file quickly and save memory.

One of the current projects for streaming response isEventsourceIt is used to improve the user’s feeling of slow speed when communicating across systems. I won’t elaborate on this.

Another is to generate a large CSV file.

When the Django process is in a web container such as gunicorn or uwsgi, if the response does not return after a certain period of time, it will be terminated by the web container. Although we can bypass this problem by prolonging the timeout of the web container, it is still a temporary solution. To fundamentally solve this problem, Python’s generator and Django framework provideStreamingHttpResponseThis streaming response is very helpful

In CSV, Chinese processing is also very important. It is necessary to ensure that CSV is opened with Excel without disorderly code.. In order to save space, I put all the code together.. The actual use shall be placed according to the project plan

Upper Code:

pythonfrom __future__ import absolute_import
import csv
import codecs
import cStringIO


class Echo(object):

    def write(self, value):
        return value

class UnicodeWriter:

    """
    A CSV writer which will write rows to CSV file "f",
    which is encoded in the given encoding.
    """

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        # Redirect output to a queue
        self.queue = cStringIO.StringIO()
        self.writer = csv.writer(self.queue, dialect=dialect, **kwds)
        self.stream = f
        self.encoder = codecs.getincrementalencoder(encoding)()

    def writerow(self, row):
        self.writer.writerow([handle_column(s) for s in row])
        # Fetch UTF-8 output from the queue ...
        data = self.queue.getvalue()
        data = data.decode("utf-8")
        # ... and reencode it into the target encoding
        data = self.encoder.encode(data)
        # write to the target stream
        value = self.stream.write(data)
        # empty queue
        self.queue.truncate(0)
        return value

    def writerows(self, rows):
        for row in rows:
            self.writerow(row)






from django.views.generic import View
from django.http.response import StreamingHttpResponse

class ExampleView(View):
    Headers = ['some', 'header']
    def get(self,request):
        Result = [['first row', 'data1'],
                  ['second row', 'data2']]
        echoer = Echo()
        writer = UnicodeWriter(echoer)
        def csv_itertor():
                yield codecs.BOM_UTF8
                yield writer.writerow(self.headers)
                for column in result:
                    yield writer.writerow(column)

        response = StreamingHttpResponse(
            (row for row in csv_itertor()),
            content_type="text/csv;charset=utf-8")
        response['Content-Disposition'
                 ] = 'attachment;filename="example.csv"'
        return response