September 12th, 2008

ZS3 teaser

Here's a peek at a library I'm working on for Amazon S3 access from Common Lisp. It uses Drakma, Closure XML, and Ironclad.

(Why make a new library at all, when there's CL-S3? Unfortunately, I had trouble working with binary data with CL-S3 on SBCL. It has been easier to make a new library than work around the trouble.)

;;; Load credentials from a file when needed
(setf *credentials* (file-credentials #p"~/.aws"))
=> #<FILE-CREDENTIALS {1003778981}>
;;; Let's make a bucket to work in.
(bucket-exists-p "zs3-demo")
=> NIL
(create-bucket "zs3-demo")
=> #<RESPONSE 200 "OK" {1004E9CFA1}>
(all-buckets)
=> #(... #<BUCKET "zs3-demo">)
(all-keys "zs3-demo")
=> #()
;;; Saved for later use
(defparameter *timestamp* (get-universal-time))
=> *TIMESTAMP*
;;; Put some boring octets...
(put-vector (octet-vector 84 118 97 114 116 195 182 109 33) "zs3-demo" "hadjaha")
=> #<RESPONSE 200 "OK" {10032F1001}>
;;; Now let's fetch them back as strings...
(get-string "zs3-demo" "hadjaha")
=> "Tvartöm!"
;;; The default for GET-STRING and PUT-STRING is UTF-8. What about
;;; other encodings?
(get-string "zs3-demo" "hadjaha" :external-format :latin1)
=> "Tvartöm!"
;;; Conditional fetching
(get-string "zs3-demo" "hadjaha" :when-modified-since *timestamp*)
=> "Tvartöm!"
(get-string "zs3-demo" "hadjaha" :unless-modified-since *timestamp*)
=> NIL
;;; Partial fetching
(get-string "zs3-demo" "hadjaha" :end 4)
=> "Tvar"
(get-string "zs3-demo" "hadjaha" :start 4)
=> "töm!"
;;; Upload a file, key name defaults to FILE-NAMESTRING of pathname
(put-file "bork.jpg" :bucket "zs3-demo" :content-type "image/jpeg")
=> #<RESPONSE 200 "OK" {1003920BA1}>
;;; Oops, it's private by default, make it publicly accessible at
;;; http://zs3-demo.s3.amazonaws.com/bork.jpg
(make-public :bucket "zs3-demo" :key "bork.jpg")
=> #<RESPONSE 200 "OK" {1004DFDE21}>
;;; Let's put up a non-public version of that file
(put-file "bork.jpg" :bucket "zs3-demo" :key "secret.jpg" :content-type "image/jpeg" :public t)
=> #<RESPONSE 200 "OK" {10031ADBA1}>
;;; Oops, non-public
(make-private :bucket "zs3-demo" :key "secret.jpg")
=> #<RESPONSE 200 "OK" {10038EE951}>
;;; secret.jpg is not accessible. But let's make 
;;; an url that will let people access it for the next 7 days
(authorized-url :bucket "zs3-demo" :key "secret.jpg" :expires (now+ (* 7 86400)) :vhost :amazon)
;;; Let's copy the secret.jpg to a third name
(copy-object :from-bucket "zs3-demo" :from-key "secret.jpg" :to-key "nej.jpg")
=> #<RESPONSE 200 "OK" {100509E791}>
;;; Compare the objects that result
(get-file "zs3-demo" "bork.jpg" "a.jpg")
=> #P"a.jpg"
(get-file "zs3-demo" "secret.jpg" "b.jpg")
=> #P"b.jpg"
(get-file "zs3-demo" "nej.jpg" "c.jpg")
=> #p"c.jpg"
(mapcar 'file-etag '("a.jpg" "b.jpg" "c.jpg"))
=> ("\"9555731bf6c416a4bd8c8b0e2700edf5\"" "\"9555731bf6c416a4bd8c8b0e2700edf5\"" "\"9555731bf6c416a4bd8c8b0e2700edf5\"")
;;; What do the first four octets of bork.jpg look like?
(get-vector "zs3-demo" "bork.jpg" :end 4)
=> #(255 216 255 224)
;;; What's in place now?
(all-keys "zs3-demo")
=> #(#<KEY "bork.jpg"> #<KEY "hadjaha"> #<KEY "nej.jpg"> #<KEY "secret.jpg">)
;;; If you want a copy to try out, email me.