Uploading Files and Handling Upload Requests in Clojure[script]

Table of Contents

img

What’s the secret?

I have used ajax uploads and interactions in Clojure forever; it’s a bread-and-butter skill to use AJAX and JSON1. However, my WordPress usage led me to believe it would be equally trivial to handle file uploads. It turned out to be less trivial than I expected, though. In summary, the secrets I needed were:

  1. Make sure the encType on the client is "multipart/form-data", which happens differently with AJAX than with plain HTML

  2. Make sure the form-data image is in the :body of the request from the front-end (in the AJAX case)

  3. Make sure the backend is configured with the right middleware to parse multipart data

    For further details about how we accomplished this, read on!

Front-end file upload input

Anyone who has done much HTML probably knows about forms that give file uploads. You have used them all the time on your social media posts or any other content-creation job. But when we got to making one of these, getting it to pop up a file selector and even being able read info about the selection through the client browser, that was easy. But getting it to go and be readable on the server turned out to be challenging.

Reagent-style ClojureScript upload form (no AJAX)

using Hiccup syntax, here is the the front-end element that does the upload. Note that there is no AJAX involved in this. The real secret to help the back-end deal with the data is the :enc-type2 of the form; without that, at best I would receive a blob data-stream without anything to help me on the server.

(defn submit-image-form
  "a form version, using a default form (no ajax)"
  []
  [:div "Submit with an html form"
   [:form {:action "/upload-image"
           :enc-type "multipart/form-data"
           :method "post"}
    [:input {:type "file"
             :name "myfileup"}]
    [:button {:type "submit"}
     "Submit image form"]]])

AJAX submissions

Now that we know how to do it with plain HTML submissions, we can better understand the solution with AJAX and re-frame. One of the advantages of this is the control you have; a common desire is to be able to display a little preview version of the image you have selected, which is not possible with a raw HTML solution.3 First, some none-obvious gotchas.

  1. First, use a generated FormData DOM element

    Hopefully you are not attempting to support the Hated Browsers4, or you’re out of luck. Note that you don’t need to specify the encoding type in this component; that is handled by the ajax request.

    (defn submit-image
  "submit the selected image that is on input `#input-id`"
  [input-id]
   (let [el (.getElementById js/document input-id)
    name (.-name el)
    file (aget (.-files el) 0)
    form-data (js/FormData.)
    _ (.append form-data "file" file)
     submit-params {:uri "/upload-image"
                    :body form-data
                    :method :post}]
     (rfc/dispatch [:upload-image submit-params])))
    
    
(defn image-selector
  "an image-selection element that shows the image you've chosen.
    
  This is a form-2 reagent component so we can avoid global state.
  https://github.com/reagent-project/reagent/blob/master/doc/CreatingReagentComponents.md#form-2--a-function-returning-a-function"
  []
  (let [UPLOADED-IMAGE (r/atom nil)
    bg-image-style (fn []
                         (if-let [i @UPLOADED-IMAGE]
                           {:background-image
                            (str "url(" i ")")}
                           {:background-color "red"}))
    input-id "image-in"]
    (fn []
      [:div.cntnr
       [:input {:type "file"
            :id input-id
            :on-change #(change-read % UPLOADED-IMAGE)
            :accept "image/*"}]
       [:div.display-image {:height "300px" :width "300px" :style (bg-image-style)}]
       [:a.button {:on-click #(submit-image input-id)}"Submit the Image"]])))

  2. Second, Get the re-frame.http-fx setup

    Here in our front-end events collection we set up our event for the image upload using re-frame-httpx5. This is the one that took me a while to get working, but that HTML solution helped me know how to phrase my problem so that I could get the answer.

    Note that certain parts of this aren’t crucial to this answer, but are necessary to make things work, such as the transit reception and the default on-success-kvec. You need some kind of response format depending what response you will actually be receiving, and you need an :on-success to tell re-frame what to do after it completes the upload, just as you want a meaningful :on-failure. Those are not necessary to this example. The hard part was getting the format to work correctly for our form-data.6 The trick is to put your form-data directly in :body instead of in, eg, params.

    (ns humforms.reframe.events
  "Master-file of re-frame dispatches, supported by fns in the events/* namespaces"
  (:require 
   [day8.re-frame.http-fx]
   [re-frame.core :refer [reg-event-fx] :as rfc]
   [ajax.core :as ajax]
   [humforms.formats :as formats]))
    
(rfc/reg-event-fx
 :upload-image
 (fn upload-image [db [_ {:keys [uri method body on-success-kvec]
                           :or {method :post
                                on-success-kvec [:on-success]}}]]
   {:http-xhrio {:method method
             :uri             uri
             :timeout         8000
                                    ;:format (ajax/text-request-format)
             :body body
             :response-format (ajax/transit-response-format
                               formats/humforms-deserialization-handlers)
             :on-success      on-success-kvec
             :on-failure      [:on-failure]}}))

Server setup using Reitit

Here I am using Reitit7 to receive our image-upload request. On the backend, you need to be sure that you are set up to handle multipart submissions, regardless of whether you are using the above HTML solution or the AJAX one. This is because forms are ALWAYS sent as multipart data, even when they have only one part.

(ns humforms.routes.admin
  "Admin routes, often for taking action on the DB, as mediated by humforms.admin.core"
  (:require
   [ring.util.http-response :as response]
   [humforms.middleware :as middleware]
   [taoensso.timbre :as log]
   [ring.middleware.params :as params]
   [reitit.ring.middleware.multipart :as multipart] ))

(defn receive-image
  "receive an image upload and handle it."
  [request]
  (log/info ">>>>> " request) ;; variations on this were helpful for understanding the request
  ;; there is actually much more information than this, but this is the pertinent part.
  ;; this data will have the same shape whether you used the HTML or AJAX solutions on the client
  ;; {:form-params {}, :query-params {}, :content-type "multipart/form-data;
  ;; :body #object[io.undertow.io.UndertowInputStream 0x7957fc1a "io.undertow.io.UndertowInputStream@7957fc1a"],
  ;; :multipart-params {"myfileup" {:filename "oogway_accidents.jpg",
  ;;                                :content-type "image/jpeg",
  ;;                                :tempfile #object[java.io.File 0xc9725c8 "/tmp/ring-multipart-6484178643718956374.tmp"],
  ;;                                :size 23783}}}

  ;; stuff would be done here, like saving the image to disk
  (response/ok "got your upload"))

(defn admin-routes
  "Admin routes"
  []
  [""
   {:middleware [middleware/wrap-base
                 middleware/wrap-formats
                 ;; query-params & form-params
                 params/wrap-params
                 ;; multipart
                 multipart/multipart-middleware]}
   ["/upload-image" {:post receive-image}]
   ])

In our projects right now we have the code that actually starts the handler and loads the various routes in its own namespace, for better or worse.

Footnotes

1 Or better yet, replace your JSON with a format that keeps your rich datatypes over the wire, like Transit!

2 React complains about the capitalization of a plain enctype element although html doesn’t care. By putting in the - Reagent will convert it to the camelCased thing that React wants.

3 I showed a pleasing little Clojurescript version of image previewing here. Note that, unlike this article subject, there is no back-end necessary to show an image preview. https://tech.toryanderson.com/2021/09/17/clojurescript-reagent-image-previewing-selector

4 From https://github.com/JulianBirch/cljs-ajax#formdata-support

Note that `js/FormData` is not supported before IE10, so if you need to support those browsers, don’t use it. `cljs-ajax` doesn’t have any other support for file uploads (although pull requests are welcome). Also note that you must include `ring.middleware.multipart-params/wrap-multipart-params` in your ring handlers as `js/FormData` always submits as multipart even if you don’t use it to submit files.

5 re-frame ajax, https://github.com/day8/re-frame-http-fx , provides a thin wrapper over the classic cljs-ajax https://github.com/JulianBirch/cljs-ajax in order to make ajax calls simpler from a re-frame perspective. I have used raw cljs-ajax before on non-reframe projects without issue, but it is the getting them to play nicely with the re-frame framework that adds complexity but also gains you more ready SPA handling of the AJAX results with on-success and on-failure helpers.

6 Of the available ajax/*-request-format, the first note was that there was not a raw-request-format, and then, nothing even mentioning what to do with form-multipart data. Raw ajax handles this by just leaving a format off and it gets auto-interpreted to the thing you need. But here, leaving the format off gives you big errors–UNLESS you have a :body key! So, there we have it.

Note that :format also needs to be specified (unless you pass :body in the map).

-https://github.com/day8/re-frame-http-fx#step-2-registration-and-use

7 Reitit is our favorite routing/handler solution because we can (and do) use it on both front-end and back-end, it is data-driven with maps and vectors instead of macro-driven like Compojure, and is highly performant. https://github.com/metosin/reitit

Tory Anderson avatar
Tory Anderson
Web App Engineer, Digital Humanist, Researcher, Computer Psychologist

Related