Friday, January 30, 2015

Upload file directly to amazone s3 bucket using html and javascript

My requirements is to upload the assets to s3 bucket. One approach is to upload file to my server and then to amazon s3, but my application deals in big file in that case it blocks my process which i don't want. Second approach i can upload directly to amazon s3 bucket in this case i can save my asset block.

I work on rails so code snippet reflects ruby but its an easy you can do in any language

Variables

AWS-BUCKET-NAME - name of the bucket in which you want to upload.
AWSAccessKeyId - Aws access key of your account.
PATH - where you want to upload file(its your logic to generate dynamic value, i used ruby SecureRandom.uuid).
REDIRECTPATH - where amazon sends the response with uploaded file path and bucket name.
Policy - it is base64 encoded policy i will explain later.
Signature - it is base64 encoded signature

HTML Code for form

AWS-BUCKET-NAME
.s3.amazonaws.com/" method="post" enctype="multipart/form-data">
  PATH">
  AWSAccessKeyId">
 
  REDIRECTPATH">
  Policy
">
  Signature">
 
 

  File to upload to S3:
 
 

 

Policy - it a base64 encoded string which contain the policy. Policy just nothing but the cross verification of the form field value(other than AWSAccessKeyId, policy and signature fields). you 3 options starts_with, exact match and range

sample policy is
example(Its ruby code but you can easy understand it) :-
def generate_encoded_policy
policy_document = {expiration: "2029-01-01T00:00:00Z",
        conditions: [
        {bucket:
AWS-BUCKET-NAME},
        ["starts-with", "$key", key_base_name],
        {acl: "private"},
        {success_action_redirect:
REDIRECTPATH},
        ["starts-with", "$Content-Type", "application/pdf"],
        ["content-length-range", 0, 104857600]
    ]
    }.to_json

    Base64.encode64(policy_document).gsub("\n","")

end 

it is a ruby function which return me the BASE64 encoded string

Signature - It is also the base64 encoded signature
example:-
def generate_signature
       Base64.encode64(
        OpenSSL::HMAC.digest(
            OpenSSL::Digest::Digest.new('sha1'),
            AWSSECRETKEY, generate_encoded_policy)
    ).gsub("\n","")
  end


for more detail refer http://aws.amazon.com/articles/1434

Friday, December 19, 2014

Create service in ubuntu to run ruby script

My Project requirement to continuously run a ruby script to do some stuff. I achieved it with nohup. If the some exception comes in my script is stop so decide to run it as a service and if service stops it should start again. 

sudo touch /etc/init/service_name.conf

Add these line to the file

description "test service by sachin choudhary"
author "sachin choudhary "


# When to start the service
start on runlevel [2345]

# When to stop the service
stop on runlevel [016]

# Automatically restart process if crashed
respawn

# Essentially lets upstart know the process will detach itself to the background
# This option does not seem to be of great importance, so it does not need to be set.
#expect fork

# Specify working directory , where your script file
chdir /home/sachin/Desktop/service

# Specify the process/command to start, e.g.
exec /home/sachin/.rvm/rubies/ruby-2.1.5/bin/ruby test.rb


And you all done :-)

NOTE:- For Ruby guys if you are using rvm and your script might require some gems, then your required gems must be installed in rvm global set else your script not load them.

Friday, November 14, 2014

Do maximum use of ri

ri is a tool that ships with Ruby. It's a companion to rdoc, allowing you to 'display information about Ruby classes, modules, and methods' from your console.

It give benefits like:
  • More Accurate results :-
    type ri new
    It will return you the list of new methods in the installed.
  • Interactive response :-
    type ri -i
    it gives you the auto complete feature.
  • Gives you more details about the class.
    start ri -i
    type String
    It will show you the detail structure including parent class, modules and who all extend it.
 If you are using rvm you have to generate ri documentation first by
rvm docs generate all

Friday, September 5, 2014

Git working, staged and commit states

There are 3 different stages in git. Detail about each stage as follows :-
  • Working :- This is the stage where all the changed files persists.
    change a file and type git status command it will show you the file as unstaged.
  • Staged :- This is the stages where the files will be surely committed in the next commit. Change some files and type command git status it will show you files in unstaged area. now type git add --all. This command will change the area of the files from working to staged. you can see by typing command git status.
  • Commit :- This is the area where files are in committed state.

Friday, July 25, 2014

Why we have to apply self to call setter method for ruby class object

class Test
  attr_accessor :name

  def set_name
    name = "some name"
  end
end

obj = Test.new
obj.set_name
obj.name  #=> nil

Why? i am calling the setter method which is already defined by attr_accessor from the method. Here the preferences comes into picture. Inside a method local variable is given more preference then to call the method, Ruby interpreter treats name as local variable. If you apply self before the method name you are specifying the interpreter to call the method.

class Test
  attr_accessor :name

  def set_name
    self.name = "some name"
  end
end

obj = Test.new
obj.set_name
obj.name #=> "some name"

Friday, June 13, 2014

Download file from Amazon S3 without ruby-sdk

Today my project requirement is to download big file from Amazone S3 bucket. I am using the ruby gem (aws-sdk). This gem provide the read method for S3 object. The disadvantage of this method is it loads the entire file in memory, and you will run short of memory. My project deals in big files so this solution doesn't works for me.

After some google i found that Amazone also provide command line utility s3cmd for s3. You can setup your IAM user account by

s3cmd --configure

It will setup Amazon S3


For s3cmd options type

s3cmd --help

Make bucket
      s3cmd mb s3://BUCKET
  Remove bucket
      s3cmd rb s3://BUCKET
  List objects or buckets
      s3cmd ls [s3://BUCKET[/PREFIX]]
  List all object in all buckets
      s3cmd la
  Put file into bucket
      s3cmd put FILE [FILE...] s3://BUCKET[/PREFIX]
  Get file from bucket
      s3cmd get s3://BUCKET/OBJECT LOCAL_FILE
  Delete file from bucket
      s3cmd del s3://BUCKET/OBJECT
  Synchronize a directory tree to S3
      s3cmd sync LOCAL_DIR s3://BUCKET[/PREFIX] or s3://BUCKET[/PREFIX] LOCAL_DIR
  Disk usage by buckets
      s3cmd du [s3://BUCKET[/PREFIX]]
  Get various information about Buckets or Files
      s3cmd info s3://BUCKET[/OBJECT]
  Copy object
      s3cmd cp s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]
  Move object
      s3cmd mv s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]
  Modify Access control list for Bucket or Files
      s3cmd setacl s3://BUCKET[/OBJECT]
  Enable/disable bucket access logging
      s3cmd accesslog s3://BUCKET
  Sign arbitrary string using the secret key
      s3cmd sign STRING-TO-SIGN
  Fix invalid file names in a bucket
      s3cmd fixbucket s3://BUCKET[/PREFIX]
  Create Website from bucket
      s3cmd ws-create s3://BUCKET
  Delete Website
      s3cmd ws-delete s3://BUCKET
  Info about Website
      s3cmd ws-info s3://BUCKET
  List CloudFront distribution points
      s3cmd cflist
  Display CloudFront distribution point parameters
      s3cmd cfinfo [cf://DIST_ID]
  Create CloudFront distribution point
      s3cmd cfcreate s3://BUCKET
  Delete CloudFront distribution point
      s3cmd cfdelete cf://DIST_ID
  Change CloudFront distribution point parameters
      s3cmd cfmodify cf://DIST_ID
  Display CloudFront invalidation request(s) status
      s3cmd cfinvalinfo cf://DIST_ID[/INVAL_ID]

Tuesday, May 6, 2014

Setup wifi driver for Broadcom (14e4:4365) on ubuntu 14.10

Setup wifi for broadcom(14e4:4365).

First check whether you have broadcom(14e4:4365) chip or not by :-

 lspci -vnn | grep 14e4

you should get the output something like 
07:00.0 Network controller [0280]: Broadcom Corporation BCM43142 802.11b/g/n [14e4:4365] (rev 01)

Now type these and you are done

sudo dpkg-reconfigure network-manager

sudo apt-get install bcmwl-kernel-source

sudo apt-get install firmware-b43-installer