Stereonaut!

curl POST data and newlines
~ Aug 15, 2010

with 4 comments

I'm writing a Rails application for the microblogging strategy for Debian (which I used to call, debian-twitter, but you know how us Debian folks are :D ). I came up with a strange issue, that at first I thought was ActionController's fault or some crap.

So, I was testing one of my controllers with something like this:

cat /tmp/mm.txt | curl -d @- http://localhost:3000/message/new

/tmp/mm.txt is a PGP signed message which then I just send as POST data to my application. So far so good. However, when accessing the data from Rails, using request.body.read, I was a getting a single line, with the newlines (carriage returns) removed. So I started looking at how ActionController was getting the HTTP data and stuff. But then I tested:

cat /tmp/mm.txt | lwp-request -m POST  http://localhost:3000/message/new

And that had the carriage returns in place.

So, I started looking at the curl man page and discovered this little gem:

–data-binary <data>

(HTTP) This posts data exactly as specified with no extra processing whatsoever.

If you start the data with the letter @, the rest should be a filename. Data is posted in a similar manner as –data-ascii does, except that newlines are preserved and conversions are never done.

If this option is used several times, the ones following the first will append data as described in -d/–data.

cat /tmp/mm.txt | curl --data-binary @- http://localhost:3000/message/new

…did indeed the tricky. Maybe someone needs this info some day.

Written by David Moreno

August 15th, 2010 at 10:10 pm

Categorized in: development

Tagged with , , ,

4 Responses to 'curl POST data and newlines'

Subscribe to comments with RSS or TrackBack to 'curl POST data and newlines'.

  1. Hey, thanks from a stranger. Had the same issue with the stripped newlines and the binary flag did the trick.

    Mumin

    19 Mar 11 at 7:28 pm

  2. Hey David, thanks! Made my day.

    Linus G Thiel

    11 Apr 11 at 8:12 am

  3. Thank you, thank you, thank you!

    André Wendt

    8 Jul 11 at 4:24 am

  4. David, you've just saved me hours of wasted time! Thanks!!

    Adam C

    19 Apr 12 at 9:28 pm

Leave a Reply