Spin Up Terraform and Chef – Part 2

April 23, 2017 Leave a comment

This the second part from this post . In this post we will cover Chef, how to write recipes, configure and test our recipes on target host managed by Vagrant.

What is chef ? chef is configuration management and automation platforms to achieve speed, scalability and consistency by turning infrastructure into flexible, human-readable, versionable and testable code.
User only need to write recipes that describe chef how to manage servers and applications and how they are configured. Chef offers :
1. Consistency

it means, we can write recipes and we can run it multiple times in different or same machine with same result.

2. Efficiency

Chef can configure thousand servers and put all configuration in one place. no more scattering code for each servers configuration

3. Scalability

Chef has capability to scale up infratructure with well manage code using roles, nodes and enviroment.

4. Reusing

We can re-use recipes and cookbooks with ease, and will produce same result.

Next, we will try to learn how we write simple configuration using chef from scratch.
We need to install the following pre-requisites installed before getting started with chef :
1. Vagrant -> For managing Virtual Machine
2. VirtualBox -> Virtual Machine from oracle
3. Ruby -> for writing recipes and resources
4. Git (optional) if want to store the code in git respository

Let start :
*Preparation*

1. Create folder for storing our recipes, let’s name it sheeps-nolegs

mkdir sheeps-nolegs && cd sheeps-nolegs

2. Create gemset , let name it chef

rvm gemset create chef
rvm gemset use chef

3. Create Gemfile file inside it, consist the following line :

source ‘https://rubygems.org’

gem ‘berkshelf’
gem ‘knife-solo’

4. Run bundle install

bundle install

knife-solo is command line tools that provides interfaces between local-chef repo and chef server.
knife-solo has 5 core commands,
1. knife solo init -> create strutured directory for chef
2. knife solo prepare -> install chef on given host
3. knife solo cook -> upload kitchen into given host and run chef-solo on it
4. knife solo bootstrap -> combination between prepare and cook
5. knife solo clean -> remove uplaoded kitchen fron given host

Berkshelf will manage cookbooks and the dependencies.

*Working With Chef*

1. Lets create chef structured folder using knife solo

knife solo init .

will produce the folllowing folder :
a. .chef -> hidden folder contains knife.rb and pem files
b. Berksfile -> Berks file contains sources for cookbooks to download
c. cookbooks -> folder to store vendor cookbooks
d. data_bags -> folder to store chef data bags
e. environments -> folder for Chef environment
f. nodes -> folder containing chef nodes
g. roles -> folder containing chef roles
h. site-cookboks -> folder to store custom cookbooks

2. In .chef folder we have file knife.rb, it consists of default chef-repo-specific configuration

a. cookbook_path -> the sub-directory for cookbooks
b. node_path -> the sub-directory for nodes
c. role_path -> the sub-directory for roles
d. environment_path >- the sub-directory for environments
e. data_bag_path -> the sub-directory for Data Bags
f. knife[:berkshelf_path] -> directory for vendoring coookbooks from Berksfile

3. Let’s try to install some cookbook into a node. in this case will try to install apache2. Edit Berksfile, and and the following line :

source “http://api.berkshelf.com”
cookbook ‘apache2’

4. Execute berks command to install apache2 cookbooks

berks install

this will install apache2 with it’s dependencies
5. then execute berks vendor cookbooks, to move cookbooks from ~/.berkshelf/cookbooks into cookbooks folder


6. Lets define a nodes.

Node represents any physical, virtual machine or clouds. Basically, this file is named as machine domain, such as sheeps.com. it consist of valid JSON configuration for specific machine.
– lets create node for sheeps.com
– vi sheeps.com.json
{
        “name”: “sheeps.com”,
        “run_list”: [
              “recipe[apache2]”
        ]
  }
– run_list is the main configuration in this file, this will contains arrays of recipes and roles.
– in sample above, it will execute recipe apache2 from apache2 cookbooks

*Vagrant*

In this post, for testing cookbooks, we will use vagrant. Vagrant is free and open-source software for creating and configuring virtual development environments.

1. Download Vagrant from this link, and follow installation instruction
2. Vagrant use base image instead of creating virtual machine from scratch, this bases known as boxes in vagrant. let’s use ubuntu 12.04 (precise64) as base image, or wen can find another boxes in vagrant cloud or https://atlas.hashicorp.com/search.

a. create vagrantfile inside our chef-solo directory
vagrant box add precise64 http://files.vagrantup.com/precise64.box –force
this will download precise64 box from repository, and option –force, to replace any existing precise64 box.
vagrant init precise64
this command will create Vagrantfile, which is typically should loook like this

b. For checking vagrant is running well invoke the following command :
vagrant up
then we can do vagrant ssh for testing ssh connection into vagrant running vm.
*notes, if password prompted, use ‘vagrant’ as default password provided for vagrant user


c. in some cases, this vm won’t have any chef client, we can install it on target vm using knife solo prepare, like we discussed above.
knife solo prepare vagrant@localhost -i ~/.vagrant.d/insecure_private_key -p 2222 -N sheeps.com
option -i to specify ssh key for machine
option -p to specify ssh port on target vm
option -N to specify which node will be used

d. now we can run our kitchen on node using the knife solo cook command
knife solo cook vagrant@localhost -i ~/.vagrant.d/insecure_private_key -p 2222 -N sheeps.com
e. by default apache2 will run on port 80, we can forward it to another port , let say 8080. add the following command into Vagrantfile :
config.vm.network :forwarded_port, guest: 80, host: 8080
f. The invoke vagrant reload, to reload our target vm
g. and try to access http://localhost:8080, if return 404, it means our apache2 has been successfully installed

End Of Part 2
In next post we will deep on how to defines roles, environment, data_bags, and create custom recipes

 

Jakarta, 23 April 2017

 

Aah Ahmad Kusumah

Advertisements
Categories: Devops, Tutorial Tags: , , ,

Spin Up Terraform and Chef – Part 1

February 24, 2017 3 comments

What is terraform ? cited from their documentation :

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

For detail feature and other terraform offer, please refer to their concise and complete documentation.

Terraform has four main component :

  1. Providers : terraform can talk to back-end service providers, such as AWS, Digital Ocean, etc
  2. Resources : resources is building blocks of terraform configuration 
  3. Variables : variables is to store all variables within terraform configuration. this will make terraform configuration will be more friendly and flexible
  4. Configuration : is *.tf extension file to store terraform configuration

This post won’t explain detail about feature, will only consist of two part :

  1. Installation and Basic configuration
  2. Test and Example to provision environment on AWS

Terraform Installation

Terraform installation is quite straight.

  1. Download installation archive here, and choose appropriate version depend on your system, in this post will demonstrate on Mac OS system. Linux could be similar.
  2. Extract downloaded zip, and copy terraform binary into folder will be set as terraform path, let say in /Users/kusumah/Documents/Development/terraform
  3. Set path , mine using .bash_profile screen-shot-2017-02-23-at-11-39-08-am
  4. Invoke command source ~/.bash_profile to update environment immediately
  5. Check installation with invoking terraform command, and if success, the output will be like this screen-shot-2017-02-23-at-11-41-48-am

Next step, will try spin up AWS server using terraform.

Terraform AWS

After finishing terraform installation, lets try to spin up simple AWS server using terraform.

  1. Create workspace directory, in this case i’ll create on /Users/kusumah/Documents/TUTORIAL/TERRAFORM/sample
  2. Create terraform configuration, lets named it as spinupserver.tf. below is my simple configuration: spinupline 2-6, we define provider will talk to. in this case, we’ll use AWS. for access_key and and secret_key can be obtained from AWS. region, define what AWS region will be used. Line 9-12, we define AWS key pair to access created AWS instance. this consist of key_name and public_key. for generating public key in linux machine, can refer this link. Line 15-21, define aws instance type.
  3. Save that file in our workspace directory
  4. Now Invoke command terraform plan , and the result when success, will look like this screen-shot-2017-02-24-at-11-35-18-am
  5. Now we can invoke terraform apply to apply the plan: result
  6. and voila now, our new instance created successfully in AWS screen-shot-2017-02-24-at-11-39-14-am
  7. To destroy current plan and terminated instance, just invoke terraform destroy, it will automatically terminated created instance defined in terraform configuration screen-shot-2017-02-24-at-11-42-44-am

 

in next post, will cover how to make terraform configuration more well structured, and we will try combine with Chef one of most popular configuration management tools among DevOps community (my opinion).

 

Jakarta, February 24th 2017

 

Aah Ahmad Kusumah

Categories: Devops, Tutorial Tags: , , ,

Grant Table User to Another User [Oracle]

August 22, 2016 Leave a comment

Below is snippet command for granting user table to another user

declare
  cursor t_name is select table_name from user_tables ;
  command varchar2(500);
begin
for c in t_name loop
command := 'GRANT SELECT, INSERT, UPDATE, DELETE ON ' ||c.table_name|| ' TO <other_user>';
   dbms_output.put_line(command);
   execute immediate command;
end loop;
end;


Jakarta, 24 August 2016

 

A. Ahmad Kusumah

Install ORACLE on AIX 6.1

July 24, 2016 Leave a comment

Oracle 11g Installation procedure on AIX system v.6.1 environment, on IBM P-series machine is summarized from best practice at one of our  client. This procedure, will be followed by Websphere Application Server Installation procedure and configuration on the same environment and machine.

The steps of Oracle Installation 11g on AIX System is briefly described as follow :

1. Please check software pre-requisite on AIX system :

  • bos.adt.base
  • bos.adt.lib
  • bos.adt.libm
  • bos.perf.libprefstat
  • bos.perf.perfstat
  • bos.perf.proctools
  • xlC.aix50.rte 8.0.0.8 or later
  • xlC.rte 8.0.0 or later

2. Run command below to verify the pre-requisite :

  • lslpp -l bos.adt.base bos.adt.lib bos.adt.libm bos.perf.perfstat bos.perf.libperfstat bos.perf.proctoolschecking

3. Change value of max number of processes allowed  :

  • Run “smit chgsys”
  • Set “Maximum number of PROCESSES allowed per user” > 2048
  • Verify value of “ARG/ENV list size in 4K byte blocks” >= 128smith

4. Create user and groups for oracle installation, using the following command:

  • mkgroup oinstall
  • mkgroup dba
  • mkgroup oper
  • useradd -g oinstall -G dba, oper -m oracle
  • passwd oracle #Set password for oracle useruseroracle

5.  Create oracle home directory and set ownership and privileges

  • mkdir -p /database/oracle/app
  • chown -R oracle:oinstall /database/oracle/app
  • chmod -R 755 /database/oracle/apphomeoracle

6. Change display setting on AIX

  • vi /home/oracle/.profile
  • add line : “DISPLAY=:1.0; export DISPLAY

7. Set ORACLE_HOME

  • vi /home/oracle/.profile
  • add the following lines :
  • ORACLE_BASE  = /opt.app/oracle
  • ORACLE_SID = orcl
  • export ORACLE_BASE
  • export ORACLE_SID
  • ORACLE_HOME = $ORACLE_BASE/product/11.1.0/db1
  • PATH = $ORACLE_HOME/bin:$PATH
  • export ORACLE_HOME
  • export PATHpathoracle

8. Switch user to oracle and run installer as usual with options -ignorePrereq

runinstaller

 

install

 

and follow installation procedure till finish …

 

Bogor, Sunday 24 July 2016

 

A. Ahmad Kusumah

 

 

 

 

 

Pointing Domain into AWS EC2

July 22, 2016 Leave a comment

Pointing domain into AWS EC2 is quite simple, what need todo are :

  1. Create EC2 instance, then write down the IP address (i.e 10.10.2.10)EC2 Management Console 2016-07-22 15-31-35
  2. Open Route 53 in AWS console (https://console.aws.amazon.com/route53/)
  3. Now you are in route 53, select DNS Management, or choose Hosted Zones in sidebar menu, then select Create Hosted ZoneRoute 53 Management Console 2016-07-22 15-35-40
  4. Type  your domain name, select type to Public Hosted Zone, then press Button CreateRoute 53 Management Console 2016-07-22 15-36-51
  5. Then Create Record Set, and create A record, and point address to your EC2 public IP in point 1Route 53 Management Console 2016-07-22 15-41-38
  6. Then write down NS from Created Hosted Zone, and change you domain name NS using NS from Hosted Zone. the changes will reflect immediatelly, but for some provider need to wait till 24 hours.Route 53 Management Console 2016-07-22 15-43-13
  7. And voila, now you have successfully point and associate your domain name into your AWS EC2 instance.

 

Jakarta, Friday 22 July 2016

 

A. Ahmad Kusumah

Setting Up Spark on OSX [el Capiten]

December 3, 2015 1 comment

Apache spark is one of fast and reliable engine for large-scale data processing.

This is simple step of installing spark on mac os, using maven. you can also using sbt to install spark.

  1. Download Apache Spark from here , you can choose any version you’d like. in this guide, i’ll pick version 1.4
  2. Screen Shot 2015-12-03 at 4.23.27 PM
  3. Unzip the downloaded file Screen Shot 2015-12-03 at 4.27.39 PM
  4. Then enter to new created folder, and invoke the following commandScreen Shot 2015-12-03 at 4.34.17 PM
  5. Then wait till process complete. it will take times depend on your internet connection., around 10 – 20 minutes.Screen Shot 2015-12-03 at 4.35.45 PM

 

And Voila, now you can use spark in your local …

 

Jakarta, 3rd December 2015

 

A. Ahmad Kusumah

Separate Multiple Console View On Eclipse

November 11, 2015 Leave a comment

This is simple stuff, but need to write for archive ..

In eclipse, we can have two console views… On the console view, you have a menu (#4) called ‘new console view’ . If we click this and select option 4 (new console view), you will see two consoles.

Screen Shot 2015-11-11 at 2.09.25 PM

 

You can select which output you want to see on each console by clicking the ‘display selected console’ button (second button from the right on the console view).

Or if you want to stick with current application log, just click ‘Pin Console’

Jakarta, 11 Nov 2015

 

Aah Ahmad Kusumah