JSON file write and upload to S3 with Nodejs

Recently I needed to upload JSON data which was written from front-end side at Amazon S3 server.
There were two requirement that I didn't know exactly.

1. Writing and reading JSON file with Nodejs.
2. Uploading JSON file to S3 Server.

After looking over how to send json data from front-end side, I'm gonna show you another step by step.

01. Send JSON data to backend side.

It's just simple example for sending data. Because we will write json file in backend server, here we sent just Object on the HTTP post body.

// for the convenience to communicate with
// HTTP protocol, I will use 'axios' library.
import axios from 'axios';

const sendJson = async () => {
 try {
  const apiUrl = 'http://YOUR_API_URL';
  const body = {
   abc: 123,
   def: 456
  };
  
  await axios.post(apiUrl, body);
 } catch (error) {
   console.error(error);
 }
};

02. Write and read JSON file.

Before uploading a file, you should make a file and read it from buffer or something else.

import fs from 'fs';

// expressjs router controller
export const upload = (req, res) => {
 const filename = 'your_file_name.json';
 const jsonString = JSON.stringify(req.body);
 
 fs.writeFileSync(filename, jsonString);
 
 const buffer = fs.readFileSync(filename);

 /// continue below
}

Use fs method 'writeFileSync' to write a file. First parameter is a file path or file name and Second is data you want to save.

And read the file by buffer to transmit it to S3 server.

03. Upload to S3 server.

You have to set AWS settings fristly then to get access key id and secret access key.


First set some configurations for S3 and make a S3 instance.

import AWS from 'aws-sdk';

// s3 instance
const s3 = AWS.S3({
 accessKeyId: 'YOUR_ID',
 secretAccessKey: 'YOUR_SECRET_KEY'
});

Second, set upload parameters and upload the file to S3.

export const upload = (req, res) => {
 // continued above...
 
 // define file destination
 const dest = 'your/destination/';
 const params = {
  Bucket: 'YOUR_BUCKET_NAME',
  Key: dest + filename,
  Body: buffer,
  ContentType: 'application/json'
 };
 
 // remove unnecessary file.
 fs.unlinkSync(filename);
 
 s3.upload(params, (err, data) => {
  if (err)  {
   throw Error(err);
  }
  
  return res.status(200).json(data);
 });
 
 // ...catch error and send error response.
}

Information for params

• Bucket: S3 bucket where you want to upload to.
• Key: File path and filename where you want to save a file.
• Body: file or data to upload.

Don't forget to remove unnecessary file after reading it by buffer. (Probably there is a way not to write and read, but to convert json into butter right away. If I find that I will share you.)