Creating simple blog with Node.js (express and mongodb)

Libraries toolkits used in this project:

  • jade
  • express
  • mongojs
  • twitter-bootstrap
  • momentjs

Let’s start creating new blog project by creating the empty express project.

express blog

The output should look like this:

create : blog
create : blog/package.json
create : blog/app.js
create : blog/public
create : blog/routes
create : blog/routes/index.js
create : blog/views
create : blog/views/layout.jade
create : blog/views/index.jade
create : blog/public/stylesheets
create : blog/public/stylesheets/style.css
create : blog/public/javascripts
create : blog/public/images

dont forget to install dependencies:
$ cd blog && npm install

Requirements for our simple blog(must contain the CRUD):

  • Add new blog posts (Create).
  • View the list of all our posts, order by latest create date (Read, there should be separate “full text” read).
  • All make mistakes, we need to make the post editable by author, it would be good to use create form (Update).
  • Remove the posts, maybe not delete but also change state to unpublished, but also provide the delete (Delete).
  • Commenting in separate post view.
  • Login for blog owner and for publishers.
The design and web structure will be created with the twitter bootstrap library, simple way to create good looking site with small amount of time.

Lets create basic structure of our mongo data model.

var post = {
subject: 'This the test subject'
, body: 'Body should contain the markdown content'
, tags: [ 'first', 'mongodb', 'express']
, created: new Date()
, modified: new Date()
, state: 'published'
, author: {
username: 'estveeb'
}
, comments: [
{
name: 'Test user'
, body: 'I like your blog'
, created: new Date()
}
]
};

Next thing is to add the simple wrapper library to communicate mongodb, called Mongojs. To start the connection to database add this to app.js and then add the posts query to your router. Run the node app.js and open browser type localhost:3000.

db = require('mongojs').connect('blog', ['post']);
...
app.get('/', function(req, res) {
  var fields = { subject: 1, body: 1, tags: 1, created: 1, author: 1 };
  db.post.find({ state: 'published'}, fields, function(err, posts) {
    if (!err && posts) {
      res.render('index.jade', { title: 'Blog list', postList: posts });
    }
  });
});


Showing individual blog post, we will be using route param pre condition which will be accessed before going through route itself. In our case we are making the request to database to make sure the post exists.

// Route param pre condition
app.param('postid', function(req, res, next, id) {
  if (id.length != 24) return next(new Error('The post id is not having correct length'));

  db.post.findOne({ _id: db.ObjectId(id) }, function(err, post) {
    if (err) return next(new Error('Make sure you provided correct post id'));
    if (!post) return next(new Error('Post loading failed'));
    req.post = post;
    next();
  });
});

app.get('/post/:postid', function(req, res) {
  res.render('show.jade', {
    title: 'Showing post - ' + req.post.subject,
    post: req.post
  });
});


In the blog post view we need to add the fields for commenting(name, comment) and handle the post request if some of the comments is posted. For security reason there is need to check if the blog post is in the post’s collections or not, in my code i haven’t added.

// Add comment
app.post('/post/comment', function(req, res) {
  var data = {
      name: req.body.name
    , body: req.body.comment
    , created: new Date()
  };
  db.post.update({ _id: db.ObjectId(req.body.id) }, {
    $push: { comments: data }}, { safe: true }, function(err, field) {
      res.redirect('/');
  });
});

The functionality without authentication is covered now we need to add the auth also, for adding new posts, editing and deleting. We will implement really basic auth system the user has username, password which are sent to server compare the hashes and give user access or not. I don’t like examples which use plain password fields so we use sha256 + salt to generate hashes.
The routes which need to have authenticated user, has middleware check for access rules.

// Login
app.get('/login', function(req, res) {
  res.render('login.jade', {
    title: 'Login user'
  });
});

app.get('/logout', isUser, function(req, res) {
  req.session.destroy();
  res.redirect('/');
});

app.post('/login', function(req, res) {
  var select = {
      user: req.body.username
    , pass: crypto.createHash('sha256').update(req.body.password + conf.salt).digest('hex')
  };

  db.user.findOne(select, function(err, user) {
    if (!err && user) {
      // Found user register session
      req.session.user = user;
      res.redirect('/');
    } else {
      // User not found lets go through login again
      res.redirect('/login');
    }

  });
});


Now the adding post functionality, which checks the user state logged in or not and then inserts data.

app.get('/post/add', isUser, function(req, res) {
  res.render('add.jade', { title: 'Add new blog post '});
});

app.post('/post/add', isUser, function(req, res) {
  var values = {
      subject: req.body.subject
    , body: req.body.body
    , tags: req.body.tags.split(',')
    , state: 'published'
    , created: new Date()
    , modified: new Date()
    , comments: []
    , author: {
        username: req.session.user.user
    }
  };

  db.post.insert(values, function(err, post) {
    console.log(err, post);
    res.redirect('/');
  });
});

In the last version I added simple editing and deleting also. In edit we do the check if article exists and then render the editor, also we can fake the POST values so there is need for additional database query before starting $set.

app.get('/post/edit/:postid', isUser, function(req, res) {
res.render('edit.jade', { title: 'Edit post', blogPost: req.post } );
});

app.post('/post/edit/:postid', isUser, function(req, res) {
db.post.update({ _id: db.ObjectId(req.body.id) }, {
$set: {
subject: req.body.subject
, body: req.body.body
, tags: req.body.tags.split(',')
, modified: new Date()
}}, function(err, post) {
res.redirect('/');
});
});


Delete functionality has to be redone as it is not good idea to delete the posts with GET which parameters come from url, also no additional check if the user clicked it by accident, so there must be also some confirm buttons.

app.get('/post/delete/:postid', isUser, function(req, res) {
  db.post.remove({ _id: db.ObjectId(req.params.postid) }, function(err, field) {
    res.redirect('/');
  });
});

I have left out many features, because the post would have gone to long, there is no editing or deleting feature, also validating data and filtering. I will add the features later with some updates to my code, any major feature is going to be covered in my blog. Check out my github page for newer implementations.

Advertisements

Creating npm package from scratch

The Node Package Manager seems to do wonderful job, it is simple to understand and has really well built up. I have created one simple IMDB scraper, which I want to add into repository so I’ll share my tips step by step.

1. I have already created directory imdb-rscraper, put all the package files in the folder, if you have many files create separate folder lib.

2. Create new file called package.json(touch package.json),this is the file where all the metadata is listed. You could basicaly use this template.

{
    "name": "imdb-rscraper",
    "author": "No Public Name here",
    "homepage": "https://veebdev.wordpress.com/",
    "version": "0.0.1",
    "description": "Scraping the data from IMDB with JQuery selectors",
    "main": "imdb-rscraper.js",
    "keywords": [
        "imdb",
        "scraper"
    ],
    "dependencies" : {
        "jsdom": ">= 0.2.12"
    },
    "repository" : {
        "type" : "git",
        "url" : ""
    }
}

3. Now make symlink the package folder run npm link. Creates link /usr/local/lib/node_modules/imdb-rscraper -> /home/risto/node/imdb-rscraper

4. Create the README file for better understanding even for yourself, give brief overview how to use your library, tool.

5. Before publishing you have to create user in repository npm adduser fill the user fields and the publish npm publish. After the last step you should see something like this. Give feedback if I missed something.

npm http PUT https://registry.npmjs.org/imdb-rscraper
npm http 201 https://registry.npmjs.org/imdb-rscraper
npm http GET https://registry.npmjs.org/imdb-rscraper
npm http 200 https://registry.npmjs.org/imdb-rscraper
npm http PUT https://registry.npmjs.org/imdb-rscraper/0.0.1/-tag/latest
npm http 201 https://registry.npmjs.org/imdb-rscraper/0.0.1/-tag/latest
npm http GET https://registry.npmjs.org/imdb-rscraper
npm http 200 https://registry.npmjs.org/imdb-rscraper
npm http PUT https://registry.npmjs.org/imdb-rscraper/-/imdb-rscraper-0.0.1.tgz/-rev/2-18de2cfa3e62cb38e8b13c84deb5db16
npm http 201 https://registry.npmjs.org/imdb-rscraper/-/imdb-rscraper-0.0.1.tgz/-rev/2-18de2cfa3e62cb38e8b13c84deb5db16
+ imdb-rscraper@0.0.1

6. Visit npmjs.org and search for your package with the declared information for each version.

 

Node.js http get example does not work here is fix

Many of us has tried the node.js example how to make HTTP get request. Only problem is that it does not get you the correct google web page as it returns the HTTP code 302. This means that the site has to redirected to other site, which is basically mapped by your IP local site google.ee (estonia) or google.fi (finland). That is the reason why the first request fails in node, I made simple workaround for this.

var http = require('http')
  , url = require('url');

function request(address) {
	http.get({ host: address, path: '/index.html'}, function(response) {
		// The page has moved make new request
		if (response.statusCode === 302) {
			var newLocation = url.parse(response.headers.location).host;
			console.log('We have to make new request ' + newLocation);
			request(newLocation);
		} else {
			console.log("Response: %d", response.statusCode);
			response.on('data', function(chunk) {
				console.log('Body ' + chunk);
			});
		}
	}).on('error', function(err) {
		console.log('Error %s', err.message);
	});
}

request('www.google.com');

Making google search

var http = require('http')
  , url = require('url');

function request(address, path) {
	http.get({ host: address, path: path}, function(response) {
		// The page has moved make new request
		if (response.statusCode === 302) {
			var newLocation = url.parse(response.headers.location).host;
			console.log('We have to make new request ' + newLocation);
			request(newLocation);
		} else {
			console.log("Response: %d", response.statusCode);
			response.on('data', function(chunk) {
				console.log('Body ' + chunk);
			});
		}
	}).on('error', function(err) {
		console.log('Error %s', err.message);
	});
}

request('www.google.com', '/search?ie=UTF-8&q=node');

Great Javascript template library Mustache

I looked for a template library as more and more data is transferred through XMLHttpRequest and the displayed html gets more difficult. First I found  JQuery template library by John Resig, the only problem is that the project seemed to be abandoned. With the release of Node.js there are now more and more javascript template libraries like Jade, EJS, Haml, and others. After googeling I found another great library called Mustache, I like it as it is easy to learn and the concept seems great.

After trying out the library I made small example how to use it.

The template.tpl is the file that contains the mustache syntax + html.

		$.get('template.tpl', function(page) {
			$.getJSON('http://api.flickr.com/services/feeds/photos_public.gne?jsoncallback=?',
				{ tags: 'cars', tagmode: 'any', format: 'json'}, function(data) {
				$('#gallery').html(Mustache.to_html(page, data));		
			});
		});

Example of how the mustache syntax would look.

{{#items}}
<div class="image">
	<div>
		<a href="{{link}}"><img src="{{media.m}}" width="240" height="180" title="{{title}}" /></a>
	</div>
</div>
{{/items}}

Full project code located on github https://github.com/riston/mustache-demo

Possible encode problems UTF-8

The common problem with fresh projects in php is that, when developing with Windows you sometimes might start programming with notepad. Well the problem arises when saving your work, because the default notepad saves your work in ANSI format. Looking the work in browser you only see strange characters and äüõü will not be displayed. To solve the problem is to save it as UTF-8 and make sure it changes the encoding.

You might see strange characters also by fetching and displaying data from database. To avoid that set the database “DEFAULT CHARACTER SET utf8” or by setting the value individually for each table. Well I did this, but the problem remains. Also set the MySQLi’s extension charset $mysqli->set_charset(“utf8”).

How does the browser know which encoding to use ? Basically browser parses the DOM tree and looks for meta tag on top with the correct charset like this

<meta http-equiv="Content-Type" content="text/html; charset=utf-8">.

Create editable data grid in PHP and JQuery

Here is small example how to create the data grid which data comes from database and it is displayed as table. Also the table must be easily editable and would not need unnecessary buttons like ‘save’ or ‘edit’. The current implementation is not very secure so you need to add your security layer in, validations, filtering and error messages.

The idea how it works is simple, the whole table is filled with input fields. When you want to edit some field hover over the field and the edit can begin, take the cursor out of the input box and the POST request is made to server for saving data(only one field at time is updated).

Features which need to be added:

  • Add new row(simply add end of table new tr and content building a bit DOM)
  • Delete row

I added the code also in https://github.com/riston. Small snippet of the index.php

<?php
$db = new mysqli('localhost', 'root', '', 'grid');
$db->set_charset('utf8');
if ($db->connect_errno) {
	die('Check the database connection again!');
}

$userQuery = 'SELECT user_id, name, age, location FROM user';
$stmt = $db->query($userQuery);
?>
<!DOCTYPE html>
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
		<link rel="stylesheet" type="text/css" href="css/style.css" />
		<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
		<script type="text/javascript">
			$(document).ready(function() {
				var textBefore = '';
				$('#grid').find('td input').hover(function() {
					textBefore = $(this).val();
					$(this).focus();
				}, function() {
					var $field = $(this),
						text = $field.val();
					$(this).blur();
					// Set back previous value if empty
					if (text.length <= 0) {
						$field.html(textBefore);
					} else if (textBefore !== text) {
						// Text has been changed make query
						var value = {
							'row': parseInt(getRowData($field)),
							'column': parseInt($field.closest('tr').children().find(':input').index(this)),
							'text': text
						};
						$.post('user.php', value)
						.error(function() {
							$('#message')
								.html('Make sure you inserted correct data')
								.fadeOut(3000)
								.html('&nbsp');
							$field.val(textBefore);
						})
						.success(function() {
							$field.val(text);
						});
					} else {
						$field.val(text);
					}

				});

				// Get the id number from row
				function getRowData($td) {
					return $td.closest('tr').prop('class').match(/\d+/)[0];
				}
			});
		</script>
		<title></title>
    </head>
    <body>
	<?php if ($stmt): ?>
	<div id="grid">
	<p id="message">Click on the field to edit data</p>
	<table>
		<thead>
			<tr>
				<th>Name</th>
				<th>Age</th>
				<th>From</th>
			</tr>
		</thead>
		<tbody>
		<?php while ($row = $stmt->fetch_assoc()): ?>
			<tr class="<?php echo $row['user_id']; ?>">
				<td><input type="text" value="<?php echo $row['name']; ?>" /></td>
				<td><input type="text" value="<?php echo $row['age']; ?>" /></td>
				<td><input type="text" value="<?php echo $row['location']; ?>" /></td>
			</tr>
		<?php endwhile; ?>
		</tbody>
	</table>
	</div>
	<?php else: ?>
		<p>No users added yet</p>
	<?php endif; ?>
    </body>
</html>

And the server XHR request handlre named user.php

<?php
// Detect if there was XHR request
if (!empty($_SERVER['HTTP_X_REQUESTED_WITH']) &&
	strtolower($_SERVER['HTTP_X_REQUESTED_WITH']) == 'xmlhttprequest') {
	$fields = array('row', 'column', 'text');
	$sqlFields = array('name', 'age', 'location');

	foreach ($fields as $field) {
		if (!isset($_POST[$field]) || strlen($_POST[$field]) <= 0) {
			sendError('No correct data');
			exit();
		}
	}

	$db = new mysqli('localhost', 'root', '', 'grid');
	$db->set_charset('utf8');
	if ($db->connect_errno) {
		sendError('Connect error');
		exit();
	}

	$userQuery = sprintf("UPDATE user SET %s='%s' WHERE user_id=%d",
			$sqlFields[intval($_POST['column'])],
			$db->real_escape_string($_POST['text']),
			$db->real_escape_string(intval($_POST['row'])));
	$stmt = $db->query($userQuery);
	if (!$stmt) {
		sendError('Update failed');
		exit();
	}

}
header('Location: index.php');
function sendError($message) {
	header($_SERVER['SERVER_PROTOCOL'] .' 320 '. $message);
}

Sublime text 2

59Avastasin antud teksti redaktori juhuslikult ning koheselt sain aru kui hea sellega on tööd teha. Siiani olen küll kasutanud (G)vimi, kuid mõned pisiasjad on härivad ning pole saanud piisavalt head lahendust või puudub oskus neid ära lahendada.

Sublime 2 saab tõmmata leheküljelt http://www.sublimetext.com/2, tegemist pole küll tasuta versiooniga kuid litsentsi ostma ei pea. Litsentsi hind on 59$ mitte ostmise puhul tüütab sind mõni kord salvestamise ajal alert window, et võimalus on osta kuid see pole väga häiriv.

Omadused mis Sublime text 2 huvitavaks teeb:

  • Mahult väike ning kiirelt avanev
  • Autocomplete sõltumata siis programeerimise keelest
  • Pluginate kirjutamine on tehtud lihtsaks(python)
  • Pluginate lisamine on kerge
  • Välimuselt on vaikimisi hea theme valitud
  • Koodi scrollimise jaoks on väike koodi minimap pandud
  • Projektide tugi, jätab meelde näiteks viimati avatud projekti kausta
  • Koodi template tugi
  • Stabiilne ning töötab kiirelt, sama ei saa väita java põhiste editoride või IDEde kohta

Pluginaid on suhteliselt palju ning nende installeerimine on võimalik ka Sublime siseselt(Install packages). Hetkel kasutan ühte pluginat, mis lihtsustab html-i kirjutamist nimega ZenCoding. Lisan pildikese, kuidas Sublime text 2 välja näeb. Kindlasti tasuks proovida seda editori, eelnevalt jäi mainimata et tegemist on cross platform editoriga võibolla asendab Notepad++ Sublime2-ga.