Mailing List Archive

How to send raw HTTP response
Hi Everybody,

is it possible for a controller method to directly produce the raw HTTP response?
My application generates a (potentially large)) ZIP file on the fly,
which I don't want to store (on disk or in memory) but rather send it
directly to the client while producing it.

In a CGI script I use Archive::Zip such as

print CGI::header(...);
$ZIP->writeToFileHandle(*STDOUT);
for my $f (@files) {
$ZIP->addFile($f, basename($f));
}

How can I achieve something similar under Catalyst?

Thanks,
Christian

--
http://www.spect-ct.com/ Molecular Imaging
http://www.invicro.com/ inviCRO
http://www.lackas.net/ Perl Delphi Linux MP3 Searchengines Domainchecker

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: How to send raw HTTP response [ In reply to ]
On Thu, Sep 25, 2008 at 11:35 AM, Christian Lackas <christian@lackas.net> wrote:
> Hi Everybody,
>
> is it possible for a controller method to directly produce the raw HTTP response?
> My application generates a (potentially large)) ZIP file on the fly,
> which I don't want to store (on disk or in memory) but rather send it
> directly to the client while producing it.
>
> In a CGI script I use Archive::Zip such as
>
> print CGI::header(...);
> $ZIP->writeToFileHandle(*STDOUT);
> for my $f (@files) {
> $ZIP->addFile($f, basename($f));
> }
>
> How can I achieve something similar under Catalyst?
>
> Thanks,
> Christian
>

You can use IO::Scalar, then pass that to $c->res->body... that's what
I do for XLS reports and it works fine, but they're not that large and
it doesn't do incremental writes.

-J

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: How to send raw HTTP response [ In reply to ]
On 25 Sep 2008, at 19:35, Christian Lackas wrote:
>
> print CGI::header(...);
> $ZIP->writeToFileHandle(*STDOUT);
> for my $f (@files) {
> $ZIP->addFile($f, basename($f));
> }
>
> How can I achieve something similar under Catalyst?

Looking at the source code for Catalyst::write, and
Catalyst::Engine::write, you need to say $c->finalize_headers, after
which just writing to STDOUT as above should do the right thing..

However, I'd have thought that clients on the other end would be
somewhat unhappy with not getting a Content-Length header...

Cheers
t0m


_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: How to send raw HTTP response [ In reply to ]
Tomas Doran <bobtfish@bobtfish.net> wrote on 09/25/2008 04:04:58 PM:

>
> On 25 Sep 2008, at 19:35, Christian Lackas wrote:
> >
> > print CGI::header(...);
> > $ZIP->writeToFileHandle(*STDOUT);
> > for my $f (@files) {
> > $ZIP->addFile($f, basename($f));
> > }
> >
> > How can I achieve something similar under Catalyst?
>
> Looking at the source code for Catalyst::write, and
> Catalyst::Engine::write, you need to say $c->finalize_headers, after
> which just writing to STDOUT as above should do the right thing..
>
> However, I'd have thought that clients on the other end would be
> somewhat unhappy with not getting a Content-Length header...

The only two consequences I know of for not including the content length is
that the download bar for the transfer lists unknown in browsers, and if
the transfer fails midway the browser does not "know".

-Wade


_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: How to send raw HTTP response [ In reply to ]
26.09.2008 00:20 Wade.Stuart@fallon.com:

> Tomas Doran <bobtfish@bobtfish.net> wrote on 09/25/2008 04:04:58 PM:
>
>>
>> On 25 Sep 2008, at 19:35, Christian Lackas wrote:
>>>
>>> print CGI::header(...);
>>> $ZIP->writeToFileHandle(*STDOUT);
>>> for my $f (@files) {
>>> $ZIP->addFile($f, basename($f));
>>> }
>>>
>>> How can I achieve something similar under Catalyst?
>>
>> Looking at the source code for Catalyst::write, and
>> Catalyst::Engine::write, you need to say $c->finalize_headers, after
>> which just writing to STDOUT as above should do the right thing..
>>
>> However, I'd have thought that clients on the other end would be
>> somewhat unhappy with not getting a Content-Length header...
>
> The only two consequences I know of for not including the content
> length is
> that the download bar for the transfer lists unknown in browsers,
> and if
> the transfer fails midway the browser does not "know".

Things will go wrong, they always do.
Not following the RFC is a very bad idea, without a content-length
header the client could react completely random.

The right way to do it would be a response with chunked transfer
encoding.

--
sebastian

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: How to send raw HTTP response [ In reply to ]
On Fri, Sep 26, 2008 at 3:14 PM, Sebastian Riedel
<sri-lists@labs.kraih.com> wrote:

>> The only two consequences I know of for not including the content length
>> is
>> that the download bar for the transfer lists unknown in browsers, and if
>> the transfer fails midway the browser does not "know".
>
> Things will go wrong, they always do.
> Not following the RFC is a very bad idea, without a content-length header
> the client could react completely random.
>
> The right way to do it would be a response with chunked transfer encoding.

If you run your app in CGI or mod_perl under Apache, the response will
be automatically turned into chunked transfer encoding by Apache if
your response header doesn't have Content-Length and the request is
made in HTTP/1.1.



--
Tatsuhiko Miyagawa

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: How to send raw HTTP response [ In reply to ]
* Tomas Doran <bobtfish@bobtfish.net> [080925 23:06]:

Hi Tomas,

thanks for the prompt reply (also to all others).

> Looking at the source code for Catalyst::write, and
> Catalyst::Engine::write, you need to say $c->finalize_headers, after
> which just writing to STDOUT as above should do the right thing..

This is supposed to print out the HTTP header immediately, right?

I tried this here:

sub download : Local {
my ($self, $c) = @_;
my ($filename, @files) = getAllfiles();
$c->response->content_type('application/zip');
$c->response->header('Content-Disposition' => "attachment; filename=$filename.zip");
$c->finalize_headers();
$c->response->{body} = undef;

my $ZIP = Archive::Zip->new();

foreach my $file (@files) {
my ($basename) = $file =~ m{.*/(.*)};
$ZIP->addFile($file, $basename);
}
$ZIP->writeToFileHandle(*STDOUT);
}

However, with this code I don't get a HTTP header at all (when run under
Catalyst server) and additionally Catalyst appends an error message
after my ZIP file.

So my questions are:
How do I tell Catalyst:
- to either send no HTTP header at all, or to force it to print the
header at a time convenient for me (e.g. just before the ZIP)?
- that I handled everything in the controller and that it should just
stop with processing the request (so nothing is send afterwards).

> However, I'd have thought that clients on the other end would be
> somewhat unhappy with not getting a Content-Length header...

As pointed out in this thread already, this mainly means that a browser
is not able to show a proper progress bar. If the ZIP is downloaded
incompletely, even though the browser may not able to report that
correctly, the uncompressing will fail with a truncation warning.

Christian


_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: How to send raw HTTP response [ In reply to ]
On Thu, Sep 25, 2008 at 08:35:14PM +0200, Christian Lackas wrote:
> Hi Everybody,
>
> is it possible for a controller method to directly produce the raw HTTP response?
> My application generates a (potentially large)) ZIP file on the fly,
> which I don't want to store (on disk or in memory) but rather send it
> directly to the client while producing it.
>
> In a CGI script I use Archive::Zip such as
>
> print CGI::header(...);
> $ZIP->writeToFileHandle(*STDOUT);
> for my $f (@files) {
> $ZIP->addFile($f, basename($f));
> }
>
> How can I achieve something similar under Catalyst?

Just call $c->write($chunk) with each chunk of data as you get it.

Catalyst will automatically send headers before the first chunk for you.

Do -not- rely on STDOUT being an appropriate filehandle to print to, that's
an accident of the engine design and could easily go away.

--
Matt S Trout Need help with your Catalyst or DBIx::Class project?
Technical Director http://www.shadowcat.co.uk/catalyst/
Shadowcat Systems Ltd. Want a managed development or deployment platform?
http://chainsawblues.vox.com/ http://www.shadowcat.co.uk/servers/

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
RE: How to send raw HTTP response [ In reply to ]
-----Original Message-----
From: Matt S Trout [mailto:dbix-class@trout.me.uk]
Sent: Saturday, September 27, 2008 13:48
To: The elegant MVC web framework
Subject: Re: [Catalyst] How to send raw HTTP response

On Thu, Sep 25, 2008 at 08:35:14PM +0200, Christian Lackas wrote:
>> Hi Everybody,
>>
>> is it possible for a controller method to directly produce the raw HTTP
response?
>> My application generates a (potentially large)) ZIP file on the fly,
>> which I don't want to store (on disk or in memory) but rather send it
>> directly to the client while producing it.
>>
>> In a CGI script I use Archive::Zip such as
>>
>> print CGI::header(...);
>> $ZIP->writeToFileHandle(*STDOUT);
>> for my $f (@files) {
>> $ZIP->addFile($f, basename($f));
>> }
>>
>> How can I achieve something similar under Catalyst?
>
>Just call $c->write($chunk) with each chunk of data as you get it.
>
>Catalyst will automatically send headers before the first chunk for you.
>
>Do -not- rely on STDOUT being an appropriate filehandle to print to, that's
an accident of the engine design >and could easily go away.
>
>--

All,

Using a method like this, how would you set the filename of the downloaded
data in the browser?

Thanks,
Chris


--
> Matt S Trout Need help with your Catalyst or DBIx::Class
project?
> Technical Director
http://www.shadowcat.co.uk/catalyst/
> Shadowcat Systems Ltd. Want a managed development or deployment platform?
>http://chainsawblues.vox.com/
http://www.shadowcat.co.uk/servers/

_



_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: How to send raw HTTP response [ In reply to ]
On Sat, Sep 27, 2008 at 04:25:51PM -0500, Chris Devine wrote:
>
> All,
>
> Using a method like this, how would you set the filename of the downloaded
> data in the browser?
>
> Thanks,
> Chris
>

You could set a Content-Disposition header, though I don't think that
is respected by all clients.

http://www.ietf.org/rfc/rfc2183.txt

Otherwise it will just use the filename found in the URL.

--
Lee Aylward
Re: How to send raw HTTP response [ In reply to ]
On 27 Sep 2008, at 19:48, Matt S Trout wrote:
> Just call $c->write($chunk) with each chunk of data as you get it.
>
> Catalyst will automatically send headers before the first chunk for
> you.
>
> Do -not- rely on STDOUT being an appropriate filehandle to print
> to, that's
> an accident of the engine design and could easily go away.

Unfortunately, if you're using Archive::Zip, you don't get an option
to get chunks you can call $c->write with, you _need_ to write to a
file handle.. (And this isn't the only module on CPAN that'd behave
like this).

Therefore, in some cases it would be extremely useful if you could
get Catalyst to generate you a file handle you could write to that
would write directly into the HTTP response, rather than having to
call $c->write. How feasible is that?

Alternatively, it would be really useful to have an
IO::WriteOnlyClosure module, so you could say something like:

my $fh = IO::WriteOnlyClosure->new(sub { $c->write(@_[0]) });
$zip->writeToFileHandle($fh);
$zip->addfile($_, basename($_) for (@files);

Thoughts / pointing out the module I missed on CPAN?

Cheers
t0m


_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: How to send raw HTTP response [ In reply to ]
Tomas Doran <bobtfish@bobtfish.net> writes:

> On 27 Sep 2008, at 19:48, Matt S Trout wrote:
>> Just call $c->write($chunk) with each chunk of data as you get it.
>>
>> Catalyst will automatically send headers before the first chunk for
>> you.
>>
>> Do -not- rely on STDOUT being an appropriate filehandle to print to,
>> that's
>> an accident of the engine design and could easily go away.
>
> Unfortunately, if you're using Archive::Zip, you don't get an option to
> get chunks you can call $c->write with, you _need_ to write to a file
> handle.. (And this isn't the only module on CPAN that'd behave like
> this).
(…)
> Thoughts / pointing out the module I missed on CPAN?

Catalyst::Response in 5.80 trunk implements ->print, just for this
puropse (I needed it for Text::CSV_XS' ->print($fh, $fields) method).

--
ilmari
"A disappointingly low fraction of the human race is,
at any given time, on fire." - Stig Sandbeck Mathisen

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/