Archive

Archive for the ‘linux’ Category

Docker Service Discovery (aka resolv the IP’s of your running containers)

september 30th, 2015 No comments

Note: In this blogpost I expect that you are aware of docker, it’s basics and how to use it. Also, don’t blame me if this doesn’t work, cause PEBKAC

I’ve been experimenting with docker for the last year in a hobby project, which is running a small Minecraft server network for a Youtuber called Xisumavoid

docker came to my attention some time last year (2014), heard people talking and praising it, so I checked it out, I’ve been hooked ever since.

One feature of docker is a lot of fun, the –link feature, this puts the hostname of the certain container to it’s proper IP.
However, this doesn’t update when the container you linked to restarts.. which is annoying.. also your container doesn’t start when that container isn’t up.

Also, if you want to connect to the containers from your host, or another host, say through a different docker container on a different host, yeah.. issues..

What I have found to work is this combination:

It’s a strange combination, but it works for me just fine.

This is how it works:

dnsdock is connected to the docker API, using the socket, Yes there is some security related discussion that discourages this practice… but for this one app, which is open source.. and has no direct incoming connection to the outside world.. I don’t worry that much.

The powerdns part, is basically a recursive dns server, which will combine the queries for all the docker instances, powerdns is quiet the powerfull dns server.. so that way I’m sure resolving doesn’t crash that easy.

Now, how do i make all of this work, like this:


#!/bin/bash
docker pull tonistiigi/dnsdock
docker pull urelx/pdns-recursor

docker run -d -v /var/run/docker.sock:/var/run/docker.sock --name dnsdock -p 172.17.42.1:5300:53/udp tonistiigi/dnsdock -environment="servername"
docker run -d --name powerdns \
-p 172.17.42.1:53:53 -p 172.17.42.1:53:53/udp \
urelx/pdns-recursor --forward-zones="servername.docker=172.17.42.1:5300, otherservername.docker=172.17.42.2:5300" --forward-zones-recurse=".=8.8.8.8"

In this example I also added a 2nd baremetal server, with ip 172.17.42.2

I’m aware this is not the most advanced thing, and could be expanded in lots of ways, but this just a simple proof of concept that does the basics you need in order to resolv containers.
More info you’ll find on the dnsdock repo.

However, how does this basically work

the hostname looks something like this:
containername.imagename.environment.docker.

so if you run a mysql server, name mysql1 (assuming you use the official mysql image) it looks like this:
mysql1.mysql.servername.docker.

Interesting fact, if you have mysql1, mysql2, etc, you can dns loadbalance:
mysql.servername.docker. will give back all the ip’s of the running containers, using that image.

 

semi-native ipv6 on telenet line for linux

december 26th, 2010 2 comments

Deze post is in het engels omdat dit toch internationaal mag gelezen worden.

So a friend of my send me a little howto how to setup semi-native ipv6 on my internet connection to my cable provider (Telenet).
Although Telenet has Docsis 3, they haven’t yet implemented the native ipv6 support.

So, the solution Thomas gave me was quick, dirty, but it works!

We’ll setup an 6to4 tunnel.
Wait would you think, isn’t this another tutorial to install an tunnel-broker like siXXs?

NOPE!

You’ll have to generate an ipv6 ip adress based on your ipv4 ipadress with the 2002: prefix
This prefix defines it’s an 6to4 semi-native ip.

So this is the script to enable ipv6 on your machine (connected to direct wan connection, so to the modem, not an router)

I got some commands from Thomas how to get 2002 ip, the rest i wrote myself.

Note: wan side needs to be on eth0 for this script, you should be able to change the script if your connection is different.

#!/bin/bash

ipv4=$(ifconfig eth0 | grep inet | cut -d : -f 2 | cut -d \  -f 1)
id="1"
ipv6=$(printf "2002:%02x%02x:%02x%02x::$id\n" `echo $ipv4 | tr "." " "`)
ip tunnel add tun6to4 mode sit remote any local $ipv4
ip link set dev tun6to4 up
ip -6 addr add $ipv6/16 dev tun6to4
ip -6 route add 2000::/3 via ::192.88.99.1 dev tun6to4 metric 1

So, why the 192.88.99.1 ip?
This is the ip of 6TO4-RELAY-ANYCAST this means, any 6to4 provider in the neighborhood of the network of your provider can give you ipv6 connectivity.
So this ip is reserved by default for this functionality.

He also send me an config example for cisco routers + to have an subnet.
I did some tuning on it to make it work dynamically.

Note: Outside (WAN) is FastEthernet0/0, inside (LAN) is FastEthernet0/1 in this example.

ipv6 unicast-routing
ipv6 general-prefix 6TO4PREFIX 6to4 FastEthernet0/0
ipv6 cef

interface Tunnel6
 description 6TO4-TUNNEL
 no ip address
 no ip redirects
 ipv6 address 6TO4PREFIX ::0:0:0:0:1/128
 ipv6 enable
 ipv6 mtu 1472
 tunnel source FastEthernet0/0 tunnel mode ipv6ip 6to4

interface FastEthernet0/1
 ipv6 address 6TO4PREFIX ::1:0:0:0:1/64
 ipv6 nd ra-interval 10
 ipv6 nd ra-lifetime 180
 ipv6 nd prefix default 180 120
 ipv6 enable
ipv6 route 2002::/16 Tunnel6
ipv6 route ::/0 2002:C058:6301::

in this case 2002:C058:6301:: is the 6to4 relay anycast ip.

Some interesting links:

http://wiki.nil.com/IPv6_over_IPv4_tunneling (some help i found to optimize cisco script)
http://www.coris.org.uk/cgi-bin/ip6addr (ipv6 adress calculator)

UPDATE: I’ve seen a lot of video’s on youtube this week of conferences, google techtalks, etc and i’ve learned a lot.

http://www.youtube.com/watch?v=AmjlptEva4Y&feature=channel they state that this 6to4 tunneling method is not the best, cause of major latency issue’s non-correct routing etc.
With this in mind, i’ll recommend, if your provider doesn’t have native v6, the time is now to start bugging them, since ip’s will run out by february 2011 and they probably just have an reserve for about 6 month’s.

Extra note: i have router-advertised both sixxs and 6to4 on my lan, and since i added the 6to4, my mac refuses to prefer ipv6 to connect, so that is an other reason to get back to brockers or get native v6.

Categories: ipv6, linux Tags: , , , , , ,

Velleman Kit K8000 and Linux

september 26th, 2010 5 comments

Dit bericht is in het engels omdat zo iedereen het kan lezen 😉

So for a while I own a Velleman k8000 kit.. and now I had the time to do something with it!
I went on a google hunt and came up with the following sites:

These sites had interesting files, but they were a little outdated.
So i started a little project to combine them and updated them to work on my ubuntu system.

The changes i made are available in github

I’m new in driver and python programming, I’m just a phper who started experimenting with python.
So if you see rookie mistakes, please, don’t flame, help and fix it, i would love to learn of it 🙂

Categories: Development, linux Tags: , , , , ,

What i have learned about Cassandra NoSQL DB

mei 12th, 2010 2 comments

NOTE: THIS IS FOR CASSANDRA 0.0.5 (OLD)

Normally I write in dutch, but because the rest of the world doesn’t understand dutch (you should learn our bastard language!), i’ll have to evolve to english (and more international) blogposting. (So don’t shoot me if i make major grammatical mistakes and don’t be a bitch about it!)

I’ll try to keep this post as up-to-date posible

So Cassandra is an NoSQL Database, that means, no SQL language.

I even dare to call it an improved memcached.

First of all i’ll post a couple of links where i learned a lot from

Documentation:

Examples:

Libraries:

There is a List of libraries on http://wiki.apache.org/cassandra/ClientOptions

I’m using the phpcassa lib. (yeah i use php, deal with it)

Since there is a lack of advanced beginner documentation, i have made an little example based on the “wtf is a supercolumn datamodel

$GLOBALS['THRIFT_ROOT'] = 'lib/classes/api/phpcassa/thrift/';
require_once $GLOBALS['THRIFT_ROOT'].'/packages/cassandra/Cassandra.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TSocket.php';
require_once $GLOBALS['THRIFT_ROOT'].'/protocol/TBinaryProtocol.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TFramedTransport.php';
require_once $GLOBALS['THRIFT_ROOT'].'/transport/TBufferedTransport.php';

include_once('lib/classes/api/phpcassa/phpcassa.php');
include_once('lib/classes/api/phpcassa/uuid.php');

CassandraConn::add_node('127.0.0.1', 9160);

/**
 * Example based on the blog example of http://arin.me/blog/wtf-is-a-supercolumn-cassandra-data-model
 */
/**
 * what you see next, you need to add to your storage-conf.xml
**/
/**
<Keyspace Name="BloggyAppy">

	<!-- CF definitions -->
	<ColumnFamily CompareWith="BytesType" Name="Authors"/>
	<ColumnFamily CompareWith="BytesType" Name="BlogEntries"/>
	<ColumnFamily CompareWith="TimeUUIDType" Name="TaggedPosts"/>
	<ColumnFamily CompareWith="TimeUUIDType" Name="Comments"
		CompareSubcolumnsWith="BytesType" ColumnType="Super"/>
	<!-- other keyspace config stuff like replication values -->

</Keyspace>
**/

/**
 * Load the ColumnFamilies
 * Normally there you should define the consistancy levels, but that depends on the number of nodes/replication factor or what kind of data it is.
 */
echo "Loading ColumnFamilies..";
$authors = new CassandraCF('BloggyAppy', 'Authors', false, 'BytesType');
$blogentries = new CassandraCF('BloggyAppy', 'BlogEntries', false, 'BytesType');
$tagged = new CassandraCF('BloggyAppy', 'TaggedPosts', false, 'TimeUUIDType');
$comments = new CassandraCF('BloggyAppy', 'Comments', true, 'TimeUUIDType', 'BytesType');
echo "Done.\n";

/**
 * Insert example data
 */

echo "Inserting data..";
$authors->insert("Arin Sarkissian", array("numPosts" => 11, "twitter" => "phatduckk", "email" => "[email protected]", "bio" => "bla bla bla"));
///
$blogentries->insert("i-got-a-new-guitar", array("title" => "This is a blog entry about my new, awesome guitar", "body" => "this is a cool entry. etc etc yada yada", "author" => "Arin Sarkissian", "tags" => "life,guitar,music", "pubDate" => time(), "slug" => "i-got-a-new-guitar"));
$timeuuid_1 = UUID::generate(UUID::UUID_TIME,UUID::FMT_STRING, "i-got-a-new-guitar");
$tagged->insert("guitar", array($timeuuid_1 => "i-got-a-new-guitar"));
$tagged->insert("life", array($timeuuid_1 => "i-got-a-new-guitar"));
$tagged->insert("music", array($timeuuid_1 => "i-got-a-new-guitar"));
$tagged->insert("__notag__", array($timeuuid_1 => "i-got-a-new-guitar"));
//
$blogentries->insert("another-cool-guitar", array("title" => "This is a blog entry about my other guitar", "body" => "this is a cool entry. etc etc yada yada", "author" => "Arin Sarkissian", "tags" => "guitar", "pubDate" => time(), "slug" => "another-cool-guitar"));
$timeuuid_2 = UUID::generate(UUID::UUID_TIME,UUID::FMT_STRING, "another-cool-guitar");
$tagged->insert("guitar", array($timeuuid_2 => "another-cool-guitar"));
$tagged->insert("__notag__", array($timeuuid_2 => "another-cool-guitar"));
//
$blogentries->insert("scream-is-the-best-movie-ever", array("title" => "This is a blog entry about my favorite movie Scream!", "body" => "this is a cool movie entry. etc etc yada yada", "author" => "Arin Sarkissian", "tags" => "movie,horror,", "pubDate" => time(), "slug" => "scream-is-the-best-movie-ever"));
$timeuuid_3 = UUID::generate(UUID::UUID_TIME,UUID::FMT_STRING, "scream-is-the-best-movie-ever");
$tagged->insert("movie", array($timeuuid_3 => "scream-is-the-best-movie-ever"));
$tagged->insert("horror", array($timeuuid_3 => "scream-is-the-best-movie-ever"));
$tagged->insert("__notag__", array($timeuuid_3 => "scream-is-the-best-movie-ever"));
///
$timeuuid_1a = UUID::generate(UUID::UUID_TIME,UUID::FMT_STRING, "Joe Blow");
$comments->insert("scream-is-the-best-movie-ever", array($timeuuid_1a => array("commenter" => "Joe Blow", "email" => "[email protected]", "comment" => "you're a dumb douche, the godfather is the best movie ever", "commentTime" => time())));
//
$timeuuid_2a = UUID::generate(UUID::UUID_TIME,UUID::FMT_STRING, "Some Dude");
$comments->insert("scream-is-the-best-movie-ever", array($timeuuid_2a => array("commenter" => "Some Dude", "email" => "[email protected]", "comment" => "be nice Joe Blow this isnt youtube", "commentTime" => time())));
//
$timeuuid_1b = UUID::generate(UUID::UUID_TIME,UUID::FMT_STRING, "Johnny Guitar");
$comments->insert("i-got-a-new-guitar", array($timeuuid_1b => array("commenter" => "Johnny Guitar", "email" => "[email protected]", "comment" => "nice axe dawg...", "commentTime" => time())));
///
echo "Done.\n";

/**
 * Fetch data
 */

echo "\n##FETCHING DATA##\n";
$taggedposts = $tagged->get("__notag__"); // You can change this value
foreach ($taggedposts as $tpost){
	echo "Fetching post: ".$tpost."\n";
	$blogentry = $blogentries->get($tpost);
	print_r($blogentry);
	if (!empty($blogentry)){
		echo "Fetching author info: \n";
		$author = $authors->get($blogentry["author"]);
		print_r($author);
		$numcomments = $comments->get_count($tpost);
		echo "Fetching ".$numcomments." comments for ".$tpost.": \n";
		$commentposts = $comments->get($tpost);
		print_r($commentposts);
	}
	echo "\n##NEXT##\n";
}
echo "\n##END FETCHING DATA##\n";

/**
 * Get a range of posts
 */
echo "\n##GET ALL BLOGENTRIES, YOU CAN ALSO LIMIT##\n";
$blogs = $blogentries->get_range();
print_r($blogs);

///
echo "\nScript Done.\n";

This is a basic working example for the use of the cassandra Basics.
Later i will post some example’s of what i have done in my own production code, like using timestamps as indexes to get some data.
I’ll also post some stuff where i was stuck etc, it should help other people, so look for my future cassandra posts ^^