IRC bot with Node.js

The old IRC protocol remains still pretty active as it is main communication for people who does not like to use Facebook. I have used IRC chat for long time and there are plenty of helpful people for discussing about programming.

Main thing is that you can connect to irc server by opening TCP connection and simply sending the commands. Lets give example with telnet, to connect you only need to:

  • telnet irc.freenode.net 6667
  • NICK username
  • USER username 8 * : Full Name
  • JOIN #node.js

As a result these command should bring you to node.js channel and you should see all the text stream. To send message to channel PRIVMSG #node.js :Test message, before sending message you have to specify who(user) or what channel receives the message. Another important thing is to follow the PING messages sent by server. If the user does not replay to the PING with PONG, user gets timeout and is disconnected from server.


All this can be really simply implemented in Node.js, the evented programming comes helpful when dealing with network. Here comes small example of IRC bot, which sends the user uptime if command @uptime is written to channel.

var net = require('net')
  , util = require('util')
  , exec = require('child_process').exec;

var options = {
    PORT: 6667
  , HOST: 'irc.freenode.net'
  , NICK: 'Miska'
  , CHANNELS: [ '#node.js', '#javascript', '#randomChan' ]
  , USER: {
        ALLOWED: [ 'risto_' ]
  }
};

var command = [
    { match: /^PING/, emit: 'pong' }
  , { match: '@uptime', emit: 'uptime' }
];

var socket = net.connect(options.PORT, options.HOST, function() {
    console.log('Connection up to irc server');
});

socket.setEncoding('ascii');
socket.setNoDelay();

socket.on('connect', function() {

    setTimeout(function() {
        socket.write(util.format('NICK %s\n', options.NICK));
        socket.write(util.format('USER %s 8 *: %s\n', options.NICK, 'Full Name'));
        socket.write(util.format('JOIN %s\n', options.CHANNELS.join(','))); 
    }, 10000);
});

socket.on('timeout', function() {
    console.log('TIMEOUT for connection');
    console.log('Total of bytes written: ' + socket.bytesWritten);
});

/**
 * Custom command for bot send the uptime
 */
socket.on('uptime', function() {
    var child = exec('uptime', function(err, stdout, stderr) {
        if (err) throw err;
        else {
            options.CHANNELS.forEach(function(elem) {
                var response = util.format('PRIVMSG %s :%s\n', elem, stdout);
                console.log(response);
                socket.write(response);
            });
        }
    });
});

socket.on('pong', function() {
    console.log('PONG message back');
    socket.write('PONG\n');
});

socket.on('data', function(data) {
    var cleanedData = data.toString().trim();
    console.log(cleanedData);

    command.forEach(function(command) {
        if (cleanedData.match(command.match)) {
            socket.emit(command.emit);
        }
    });
});



Classical and ES5 JavaScript inheritance

The inheritance and OOP concept is in JavaScript really hard to learn because there is not any standards for that like in Java or PHP. There is no structure like class, the keyword is in strict mode reserved keyword but no implementation or use. In JavaScript there are functions and using prototype to implement some of the OOP conception. That is why I made small example how to use the JavaScript objects in classical and ES5 way. This is not the complete example of OO.

Classical example and using the NEW keyword:

(function(document) {
'use strict';

  var Shape = function() { };
  
  Shape.prototype = {
    x: 30,
    y: 30,
    width: 25,
    height: 25,
    
    draw: function(ctx) {
      throw new Error("This is parent method");
    },
    
    print: function() {
      console.log(this.x + ':' + this.y + ' ' + this.width + '<>' + this.height);
    }
  };
  
  var Circle = function() { this.radius = 20; };
  Circle.prototype = new Shape();
  Circle.prototype.draw = function(ctx) {
    ctx.arc(12, 22, 66, 0, 2 * Math.PI, false);
  };
  
  var Square = function() { };
  Square.prototype = new Shape();
  Square.prototype.draw = function(ctx) {
    ctx.fillRect(this.x, this.y, this.width, this.height);
  };  
    
  var canvas = document.getElementById('panel');
  if (canvas.getContext) {
    var ctx = canvas.getContext('2d');
    var perfectCircle = new Circle();
    var mySquare = new Square();
    
    mySquare.draw(ctx);
    mySquare.print();
    
    perfectCircle.draw(ctx);
    perfectCircle.print();
    
  }
  
})(document);

ES5 example:

(function(document, console) {
'use strict';

  var Shape = Object.create({
    x: 30,
    y: 30,
    width: 25,
    height: 25,
    
    draw: function(ctx) {
      throw new Error("This is parent method");
    },
    
    print: function() {
      console.log(this.x + ':' + this.y + ' ' + this.width + '<>' + this.height);
    }
  });

 
  var Circle = Object.create(Shape);
  Circle.radius = 10;
  Circle.draw = function(ctx) {
    ctx.beginPath();
    ctx.arc(this.x, this.y, this.radius, 0, 2 * Math.PI, false);
    ctx.stroke();
  };
  
  var Square = Object.create(Shape);
  Square.draw = function(ctx) {
    ctx.fillRect(this.x, this.y, this.width, this.height);
  };  
  
  // Drawing to canvas
  var canvas = document.getElementById('panel');
  if (canvas.getContext) {
    var ctx = canvas.getContext('2d');
    var perfectCircle = Object.create(Circle);
    var mySquare = Object.create(Square);
    
    mySquare.draw(ctx);
    mySquare.print();
    
    perfectCircle.draw(ctx);
    perfectCircle.print();
    
  }
  
})(document, console);

Also the html to get the example working:

<!DOCTYPE html>
<html>
<head>
<meta charset=utf-8 />
<title>Inheritance example</title>
</head>
<body>
  <canvas id="panel" width="150" height="150">Canvas not supported get better browser</canvas>
</body>
</html>

Result image should be something like this(well only the shapes not gradient):

jQuery too slow ?

jQuery is great JavaScript library, only that in time the core code has changed too heavy and most of the functionality is not used. Also jQuery has great support for cross browser functions, which need additional checks and boilerplate. This all slows down performance and may not be suitable for the mobiles.

What could be alternatives is to use some of the lightweight versions of jQuery like zepto.js, snack.js etc. Well even if these libraries seems to be too heavy, the last thing is to use JavaScript without any library or create your own layer top of the functions you need.

I did some of the performance test with jQuery, Zepto.js and using pure JavaScript DOM. All of the tests are done with , which runs the testing code in loop and counts the operations per second. ALl the benchmark tests are pushed to Github.

Example code of the JSlitmus test:

/*
 * Simple DOM object creation without adding to html.
 */
JSLitmus.test('DOM element creation', function() {
  document.createElement('div');
});


As you can see the winner of benchmark is JavaScript DOM API, also if the development is clearly mobile platform you should take advantage from lighter libraries.

Expressive way to handle errors in expressjs

When you have created new express project, the errors get handled a bit differently in dev mode and in production mode. In development mode the errors are show in express error template with some error stack if showStack is set to true. But in production mode you will be shown as less as you need the ‘Internal Server Error’ on plain text page. Here is our plan to make the production side errors more fancier and keep the development side informative.

First take of the production mode default error handling.

app.configure('production', function(){
  //app.use(express.errorHandler()); 
});

Add app.error which will be run on if error is caught. I have already created views for errors which needs some customization on jade template.

app.error(function(err, req, res, next){
  if (err instanceof NotFound) {
    res.render('error/404.jade', { title: 'Not found 404' });
  } else {
    res.render('error/500.jade', { title: 'Error', error: err });
  }
});

And in the end of app.js, added the /* router which routes all the unknown pages to 404 page by throwing NotFound errror. So by going to url like this localhost:3000/path/that/does/not/exist does not lead you to the some GET that route does not exist, but into 404 page.

// The 404
app.get('/*', function(req, res){
    throw new NotFound;
});

function NotFound(msg){
    this.name = 'NotFound';
    Error.call(this, msg);
    Error.captureStackTrace(this, arguments.callee);
}

Running now the server in production mode NODE_ENV=production node app.js, should give on visiting unknown page this:

And on the 500 error which are thrown when user tries to go in /add/post page without permission:

This is one way to handle the errors in expressjs, if you know any better methods feel free to post. The code is also added into github.

Creating simple blog with Node.js: flash messages

One way to give user feedback is to use the flash messages, after certain action you display information. Example user deletes the post and is redirected to main page, now display the information if all were successful or not.

To make use of the flash messages you need to add the cookieParser

app.configure(function(){
  app.set('views', __dirname + '/views');
  app.set('view engine', 'jade');
  app.use(express.logger());
  app.use(express.bodyParser());
  app.use(express.methodOverride());
  app.use(express.cookieParser());
  app.use(express.session({ secret: 'wasdsafeAD' }));
  app.use(app.router);
  app.use(express.static(__dirname + '/public'));
});

This is the way to add information to flash messages, you can acces the object by req.flash().

if (!err) {
  req.flash('info', 'Post has been sucessfully edited');
}

To use it in all the view so I would not have to pass the flash object into every render function, is to register helper.

app.dynamicHelpers({
...
  flash: function(req, res) {
    return req.flash();
  }
});

Also i have added into layout.jade before header partial !=partial(‘alert’, flash) and in view partial is the message look.

mixin showAlert(alerts)
    for alert in alerts
      div #{alert}

- if (flash.info)
  div.alert.alert-info
    mixin showAlert(flash.info)

- if (flash.success)
  div.alert.alert-success
    mixin showAlert(flash.success)

- if (flash.error)
  div.alert.alert-error
    mixin showAlert(flash.error)

Select box and other choice

Sometimes there is need for dynamical forms where in select box you got the other choice which needs to trigger out the textbox. Only the problem is that this would create new POST field with different name and this needs more checks and ifs on the server side.

Here is one simple way to solve this, when user picks from list ‘other’ we show additional textbox but in the same time change the form names also so if the other is chosen then the value from textbox is posted and the select box uses temporary name suffixed ‘-input’. Added the jsfiddle address for demo http://jsfiddle.net/staar2/Hsq3j/

(function() {
    var name = 'Grizly'
      , $select = $('select[name="' + name + '"]')
      , nameAttr = $select.attr('name');
    
    $select.change(function(e) {
        if ($(this).val() === 'Other') {
            $select.attr('name', nameAttr + '-input');
            $select.after($('<input>', {
                'type': 'text',
                'name': nameAttr
            }));            
        } else {
            $('select[name="' + name + '-input"]')
                .attr('name', name);
            $('input[name="' + name + '"]').remove();
        }         
    });        
})();​
<form action="" method="post">
    
<select name="Grizly">
    <option>First</option>
    <option>Second</option>
    <option>Third</option>
    <option>Fourth</option>
    <option>Other</option>
</select>
<input type="submit" name="send" value="Send" />
</form>
​

Creating simple blog with Node.js (express and mongodb)

Libraries toolkits used in this project:

  • jade
  • express
  • mongojs
  • twitter-bootstrap
  • momentjs

Let’s start creating new blog project by creating the empty express project.

express blog

The output should look like this:

create : blog
create : blog/package.json
create : blog/app.js
create : blog/public
create : blog/routes
create : blog/routes/index.js
create : blog/views
create : blog/views/layout.jade
create : blog/views/index.jade
create : blog/public/stylesheets
create : blog/public/stylesheets/style.css
create : blog/public/javascripts
create : blog/public/images

dont forget to install dependencies:
$ cd blog && npm install

Requirements for our simple blog(must contain the CRUD):

  • Add new blog posts (Create).
  • View the list of all our posts, order by latest create date (Read, there should be separate “full text” read).
  • All make mistakes, we need to make the post editable by author, it would be good to use create form (Update).
  • Remove the posts, maybe not delete but also change state to unpublished, but also provide the delete (Delete).
  • Commenting in separate post view.
  • Login for blog owner and for publishers.
The design and web structure will be created with the twitter bootstrap library, simple way to create good looking site with small amount of time.

Lets create basic structure of our mongo data model.

var post = {
subject: 'This the test subject'
, body: 'Body should contain the markdown content'
, tags: [ 'first', 'mongodb', 'express']
, created: new Date()
, modified: new Date()
, state: 'published'
, author: {
username: 'estveeb'
}
, comments: [
{
name: 'Test user'
, body: 'I like your blog'
, created: new Date()
}
]
};

Next thing is to add the simple wrapper library to communicate mongodb, called Mongojs. To start the connection to database add this to app.js and then add the posts query to your router. Run the node app.js and open browser type localhost:3000.

db = require('mongojs').connect('blog', ['post']);
...
app.get('/', function(req, res) {
  var fields = { subject: 1, body: 1, tags: 1, created: 1, author: 1 };
  db.post.find({ state: 'published'}, fields, function(err, posts) {
    if (!err && posts) {
      res.render('index.jade', { title: 'Blog list', postList: posts });
    }
  });
});


Showing individual blog post, we will be using route param pre condition which will be accessed before going through route itself. In our case we are making the request to database to make sure the post exists.

// Route param pre condition
app.param('postid', function(req, res, next, id) {
  if (id.length != 24) return next(new Error('The post id is not having correct length'));

  db.post.findOne({ _id: db.ObjectId(id) }, function(err, post) {
    if (err) return next(new Error('Make sure you provided correct post id'));
    if (!post) return next(new Error('Post loading failed'));
    req.post = post;
    next();
  });
});

app.get('/post/:postid', function(req, res) {
  res.render('show.jade', {
    title: 'Showing post - ' + req.post.subject,
    post: req.post
  });
});


In the blog post view we need to add the fields for commenting(name, comment) and handle the post request if some of the comments is posted. For security reason there is need to check if the blog post is in the post’s collections or not, in my code i haven’t added.

// Add comment
app.post('/post/comment', function(req, res) {
  var data = {
      name: req.body.name
    , body: req.body.comment
    , created: new Date()
  };
  db.post.update({ _id: db.ObjectId(req.body.id) }, {
    $push: { comments: data }}, { safe: true }, function(err, field) {
      res.redirect('/');
  });
});

The functionality without authentication is covered now we need to add the auth also, for adding new posts, editing and deleting. We will implement really basic auth system the user has username, password which are sent to server compare the hashes and give user access or not. I don’t like examples which use plain password fields so we use sha256 + salt to generate hashes.
The routes which need to have authenticated user, has middleware check for access rules.

// Login
app.get('/login', function(req, res) {
  res.render('login.jade', {
    title: 'Login user'
  });
});

app.get('/logout', isUser, function(req, res) {
  req.session.destroy();
  res.redirect('/');
});

app.post('/login', function(req, res) {
  var select = {
      user: req.body.username
    , pass: crypto.createHash('sha256').update(req.body.password + conf.salt).digest('hex')
  };

  db.user.findOne(select, function(err, user) {
    if (!err && user) {
      // Found user register session
      req.session.user = user;
      res.redirect('/');
    } else {
      // User not found lets go through login again
      res.redirect('/login');
    }

  });
});


Now the adding post functionality, which checks the user state logged in or not and then inserts data.

app.get('/post/add', isUser, function(req, res) {
  res.render('add.jade', { title: 'Add new blog post '});
});

app.post('/post/add', isUser, function(req, res) {
  var values = {
      subject: req.body.subject
    , body: req.body.body
    , tags: req.body.tags.split(',')
    , state: 'published'
    , created: new Date()
    , modified: new Date()
    , comments: []
    , author: {
        username: req.session.user.user
    }
  };

  db.post.insert(values, function(err, post) {
    console.log(err, post);
    res.redirect('/');
  });
});

In the last version I added simple editing and deleting also. In edit we do the check if article exists and then render the editor, also we can fake the POST values so there is need for additional database query before starting $set.

app.get('/post/edit/:postid', isUser, function(req, res) {
res.render('edit.jade', { title: 'Edit post', blogPost: req.post } );
});

app.post('/post/edit/:postid', isUser, function(req, res) {
db.post.update({ _id: db.ObjectId(req.body.id) }, {
$set: {
subject: req.body.subject
, body: req.body.body
, tags: req.body.tags.split(',')
, modified: new Date()
}}, function(err, post) {
res.redirect('/');
});
});


Delete functionality has to be redone as it is not good idea to delete the posts with GET which parameters come from url, also no additional check if the user clicked it by accident, so there must be also some confirm buttons.

app.get('/post/delete/:postid', isUser, function(req, res) {
  db.post.remove({ _id: db.ObjectId(req.params.postid) }, function(err, field) {
    res.redirect('/');
  });
});

I have left out many features, because the post would have gone to long, there is no editing or deleting feature, also validating data and filtering. I will add the features later with some updates to my code, any major feature is going to be covered in my blog. Check out my github page for newer implementations.

Creating npm package from scratch

The Node Package Manager seems to do wonderful job, it is simple to understand and has really well built up. I have created one simple IMDB scraper, which I want to add into repository so I’ll share my tips step by step.

1. I have already created directory imdb-rscraper, put all the package files in the folder, if you have many files create separate folder lib.

2. Create new file called package.json(touch package.json),this is the file where all the metadata is listed. You could basicaly use this template.

{
    "name": "imdb-rscraper",
    "author": "No Public Name here",
    "homepage": "http://veebdev.wordpress.com/",
    "version": "0.0.1",
    "description": "Scraping the data from IMDB with JQuery selectors",
    "main": "imdb-rscraper.js",
    "keywords": [
        "imdb",
        "scraper"
    ],
    "dependencies" : {
        "jsdom": ">= 0.2.12"
    },
    "repository" : {
        "type" : "git",
        "url" : ""
    }
}

3. Now make symlink the package folder run npm link. Creates link /usr/local/lib/node_modules/imdb-rscraper -> /home/risto/node/imdb-rscraper

4. Create the README file for better understanding even for yourself, give brief overview how to use your library, tool.

5. Before publishing you have to create user in repository npm adduser fill the user fields and the publish npm publish. After the last step you should see something like this. Give feedback if I missed something.

npm http PUT https://registry.npmjs.org/imdb-rscraper
npm http 201 https://registry.npmjs.org/imdb-rscraper
npm http GET https://registry.npmjs.org/imdb-rscraper
npm http 200 https://registry.npmjs.org/imdb-rscraper
npm http PUT https://registry.npmjs.org/imdb-rscraper/0.0.1/-tag/latest
npm http 201 https://registry.npmjs.org/imdb-rscraper/0.0.1/-tag/latest
npm http GET https://registry.npmjs.org/imdb-rscraper
npm http 200 https://registry.npmjs.org/imdb-rscraper
npm http PUT https://registry.npmjs.org/imdb-rscraper/-/imdb-rscraper-0.0.1.tgz/-rev/2-18de2cfa3e62cb38e8b13c84deb5db16
npm http 201 https://registry.npmjs.org/imdb-rscraper/-/imdb-rscraper-0.0.1.tgz/-rev/2-18de2cfa3e62cb38e8b13c84deb5db16
+ imdb-rscraper@0.0.1

6. Visit npmjs.org and search for your package with the declared information for each version.

 

Great Javascript template library Mustache

I looked for a template library as more and more data is transferred through XMLHttpRequest and the displayed html gets more difficult. First I found  JQuery template library by John Resig, the only problem is that the project seemed to be abandoned. With the release of Node.js there are now more and more javascript template libraries like Jade, EJS, Haml, and others. After googeling I found another great library called Mustache, I like it as it is easy to learn and the concept seems great.

After trying out the library I made small example how to use it.

The template.tpl is the file that contains the mustache syntax + html.

		$.get('template.tpl', function(page) {
			$.getJSON('http://api.flickr.com/services/feeds/photos_public.gne?jsoncallback=?',
				{ tags: 'cars', tagmode: 'any', format: 'json'}, function(data) {
				$('#gallery').html(Mustache.to_html(page, data));		
			});
		});

Example of how the mustache syntax would look.

{{#items}}
<div class="image">
	<div>
		<a href="{{link}}"><img src="{{media.m}}" width="240" height="180" title="{{title}}" /></a>
	</div>
</div>
{{/items}}

Full project code located on github https://github.com/riston/mustache-demo

Objektid

Loome objekti inimese.

Objektide loomisel kasutatakse { } samas aga massiivi loomisel tuleb kasutada [ ]


var inimene = {
 nimi : 'Miku',
 vanus : 25,
 elukoht : 'Sabataguse',
 hobid : ['suustamine', 'kudumine', 'mahla joomine'],
 ütle : function() {
 alert(this.nimi);
 }
};

alert(inimene.nimi);
alert(inimene.hobid.join('-'));
inimene.ütle();

Objekti loomine kasutamine funktsiooni konstruktorit, mis annab võimaluse luua erinevaid objekte.


function Inimene(nim) {
 this.nimi = nim;
 this.vanus = 25;
 this.ütle = function() {
 alert(this.nimi + ' ' + this.vanus);
 }
}

var madis = new Inimene('Madis');
// Muudame vanuse
madis.vanus = 33;
madis.ütle();

var maila = new Inimene('Maila');
maila.vanus = 19;
maila.ütle();