Lost sleep over JSON and Rack::PostBodyContentTypeParser

I’ve been fighting this issue the last couple nights. I wrote earlier about how Rack::PostBodyContentTypeParser can automagically turn a posted JSON object into a Rack / Sinatra params hash. So, I wrote some tests to make sure this was the case and moved on. Well, it turns out in real life things weren’t working and I couldn’t figure out why. Everything looked cool, but the hash wasn’t getting set when I did an AJAX call in the browser – everything was empty. I looked at everything, from the server, to the JS library, to the browser, to setting different content types in prototype.js etc… UGH!

The short of it is that Rack::PostBodyContentTypeParser requires exactly application/json in order to automagically turn the posted JSON object into Rack params and prototype.js (and jquery.js were adding an encoding type of charset=UTF-8 so the entire header entry was coming across as this CONTENT_TYPE: application/json; charset=UTF-8. So, as a fix, I’m just including the Rack::PostBodyContentTypeParser in the Sinatra application with one small change. Here’s the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

module Rack
 
  # A Rack middleware for parsing POST/PUT body data when Content-Type is
  # not one of the standard supported types, like <tt>application/json</tt>.
  #
  # TODO: Find a better name.
  #
  class PostBodyContentTypeParser
 
    # Constants
    #
    CONTENT_TYPE = 'CONTENT_TYPE'.freeze
    POST_BODY = 'rack.input'.freeze
    FORM_INPUT = 'rack.request.form_input'.freeze
    FORM_HASH = 'rack.request.form_hash'.freeze
 
    # Supported Content-Types
    #
 
    ################## turned into regex so it matches type with encoding data...
    #APPLICATION_JSON = 'application/json'.freeze
    APPLICATION_JSON = /^application\/json/.freeze
 
    def initialize(app)
      @app = app
    end
 
    def call(env)
      case env[CONTENT_TYPE]
      when APPLICATION_JSON
        env.update(FORM_HASH => JSON.parse(env[POST_BODY].read), FORM_INPUT => env[POST_BODY])
      end
      @app.call(env)
    end
 
  end
end

I tested that this worked by writing the following:

1
2
3
4
5
6
7
8
9
10

def test_post_as_json_converts_to_params
  # sanity check that post with normal params works...
  post '/test_params_as_json', :param1=>"param one"
  assert_equal last_response.body,"params[:param1]=param one"
  post '/test_params_as_json', {:param1=>"param one"}.to_json, "CONTENT_TYPE"=>"application/json"
  assert_equal last_response.body,"params[:param1]=param one"
  # this is the problem, adding a charset to the content type seems to breaks rack-contrib/post_body_content_type_parser.rb
  post '/test_params_as_json', {:param1=>"param one"}.to_json, "CONTENT_TYPE"=>"application/json; charset=UTF-8"
  assert_equal last_response.body,"params[:param1]=param one"    
end
Posted: April 4th, 2010 | Author: jay | Filed under: Code | Tags: , , , , , , , , | 4 Comments »

4 Comments on “Lost sleep over JSON and Rack::PostBodyContentTypeParser”

  1. 1 Michael Koukoullis said at 12:57 am on August 2nd, 2010:

    I tracked down the same issue today. FireFox was the culprit in my case. Safari and Chrome don’t append the utf-8 charset stuff.

    Monkey patching the parser and including it manually in my sinatra app.

  2. 2 Testing POST with Rack::Test - Software Warlock said at 2:17 pm on December 18th, 2010:

    [...] figuring out how to specify JSON post data in your POST tests is not immediately obvious. This article by Jay Wiggins got me on the right track, and I though I’d [...]

  3. 3 Tim Lucas said at 7:32 pm on April 27th, 2011:

    For those looking for a fix, this fork/pull request has it:
    https://github.com/rack/rack-contrib/pull/22

  4. 4 Matt Todd said at 9:08 pm on April 29th, 2011:

    Good catch and thanks for the patch. I merged it in tonight.

    Cheers!

Leave a Reply