Creating your networks

Goal is to create a Tenant Network where the Instances will be connected. Create an External Network that is connected to the Internet. Create a Router that connects both these networks to give your instances access to the Internet.

In the Admin tab, create a Network for the project/tenant. Do not select the External Network, since this is a Tenant Network.

Create a Subnet for the Tenant Network. This uses a private address space, so you can use any Network address you choose.

In the Subnet Detail Tab, enable DHCP for your VMs to get DHCP IP addresses, and also setup the DNS servers.

Create an External Network that will provide Internet Access to your VM Instances. Select External Network here since this is an External Network.

Create an External Subnet for the External Network. My External Subnet is 10.112.252.0/24. Although this is a Private Address, this address is controlled by my Enterprise Networking Team and Routable internally within the Enterprise Intranet. This address will be NAT'ed at my Enterprise Network Edge when going to the Internet. This is the Network address I am using on my eth0 interfaces. The External Bridge br-ex that we created in the previous post, is going to bridge the Openstack traffic to the External Network. The Gateway is which ever gateway you use in your network.

Configure the Subnet Detail for your External Subnet. Disable DHCP for your External Network. Most probably another DHCP server is running and your eth0s are getting the IP address from it. In any case you don't need to run DHCP server on this network. You will need to setup your IP allocation pool for this External Subnet. If you have some Static IPs requested from your Network Administrator, you can use that for your pool.

Create a Router for the Tenant.

Set the Gateway for the Router to External Network that was created before. Also, add an interface to the router that is connected to the Tenant Private Network that was created before.

Now, your network topology will look like this.

Your instance is connected to the Tenant Network. The router connects the Tenant Network to the External Network. The router does the NATing to give your instances the access to the Internet. In this case the 192.168.1.0/24 address is NAT'ed to the 10.112.252.0/24 address. Again this will be NATed at the Enterprise Intranet Edge to a real Public IP before it goes to the Internet. In my setup there is a double NAT happening. If your external network has Internet Routable addresses then there will be only one NAT happening.

Add rules to the Default Security Group to allow ICMP, TCP and UDP traffic to the instances.

Check if your instances can ping the addresses in the Internet.

Now your instances can access the Internet. If your instances need to be accessed from the Internet, because you are running a Web server that needs be accessed from the Internet, then you need to associate a floating-ip to the instance. The floating-ip's are allocated from the Static IP pool that was configured with the External Subnet. Allocate a floating-ip from the pool, and then associate it to the instance. Then your instance will be accessible from the Internet.

Now SSH to your instance from outside. In my case I am still within the Enterprise Intranet. If your External Network is part of your DMZ, and you have addresses within that DMZ range, then your instances will be accessible from outside the Enterprise Firewall.

Setting up the L3 Agent and getting your Virtual Machine Instances to talk to the Internet

The Rackspace Private Cloud cookbooks setup the Openstack deployment without setting up the L3 Agent. The L3 Agent is not installed. The L3 Agent is required to route the Network Traffic to the Internet. By default the Openstack Neutron Networking deployment will be setup with the Virtual Machine Instances in Isolated Networks, in all three modes: Flat, VLAN, GRE. You can create multiple private networks for projects and create routers to route traffic between these private networks, but the VMs cannot reach the Internet. This the Rackspace Private Cloud team did because the L3 Agent is not fully supported in High Availability (HA) deployment modes.

I am not setting up HA. GRE mode is quite easy to setup because the configuration required in the physical network is minimal, compared to VLAN networking.

This is my setup:

To setup the L3 Agent follow the steps below:

Step1: On the Network Node Install the L3 agent

# apt-get -y install quantum-l3-agent

Step2: Enable IP_Forwarding on the Compute and the Network Node as below

# sed -i 's/#net.ipv4.ip_forward=1/net.ipv4.ip_forward=1/' /etc/sysctl.conf 
# sysctl net.ipv4.ip_forward=1

Step3: On the Network Node create the external bridge br-ex that will be used to access the Internet

# ovs-vsctl add-br br-ex

Step4: For Step4 you need to have console access to the Network Node. SSH will not do. This is because you will be reconfiguring the eth0 interface that is used for SSH and Management traffic. This is like chopping the branch you are sitting on if you use SSH.

Modify your eth0 configuration in /etc/network/interfaces file to look like this:

auto eth0
iface eth0 inet manual
up ifconfig $IFACE 0.0.0.0 up
up ip link set $IFACE promisc on
down ip link set $IFACE promisc off
down ifconfig $IFACE down

Move your IP configuration you had on eth0 to the br-ex interface:

auto br-ex
iface br-ex inet static
address 10.112.252.245
netmask 255.255.255.0
gateway 10.112.252.253
dns-nameservers 10.112.116.138 10.112.116.139 10.112.64.1

Add the eth0 interface to br-ex bridge.

# ovs-vsctl add-port br-ex eth0

Reboot the Network Node. You should get your connectivity back to the Network Node for SSH.

Step5: As a troubleshooting step restart all Quantum services on the Network Node:

# cd /etc/init.d/; for i in $( ls quantum-* ); do sudo service $i restart; done

Using GRE based Neutron Networking with Rackspace Private Cloud

If you have used the environment file that was given in the previous post, then we are setting up Neutron/Quantum Networking. By default Rackspace Private Cloud cookbooks setup VLAN based networks. To setup a GRE network, you must also add a network_type attribute in your environment file.

"override_attributes": {
    "nova": {
        "network": {
            "provider": "quantum"
        }
    },
    "quantum": {
        "ovs": {
            "network_type": "gre"
        }
    }
}

The environment file will now look as below:

root@chef-workstation:~/chef-cookbooks/environments# cat grizzly-quantum-1.json 
{
  "name": "grizzly-quantum-1",
  "description": "Chef-server Grizzly Quantum Multinode environment",
  "json_class": "Chef::Environment",
  "chef_type": "environment",
  "override_attributes": {
    "nova": {
      "libvirt": { "virt_type": "qemu" },
      "network": {
        "provider": "quantum"
      }
    },
    "quantum": {
         "ovs": {
              "network_type": "gre"
         }
    }, 
    "mysql": {
      "allow_remote_root": true,
      "root_network_acl": "%",
      "server_root_password": "fr3sca",
      "server_debian_password": "fr3sca"
    },
    "osops_networks": {
      "nova": "10.112.252.0/24",
      "public": "10.112.252.0/24",
      "management": "10.112.252.0/24"
    }
  }
}

After making this change to the environment file, upload the environment file to the Chef-Server:

# knife environment from file grizzly-quantum-1.json

Now run the chef-client on the nodes: Controller, Network and Compute. Once the chef-client runs on the nodes, the Neutron Networking will be setup to use GRE instead of VLAN for Networking.

Log into the Horizon Dashboard

If you had any errors and chef-client terminated, just try re-running the chef-client on the nodes. Sometimes due to network latency or other issues during installation of all the packages, the chef-client encounters errors.

Once the chef-client has successfully completed on all the nodes, then do the following step. Run the following command on the network and compute nodes to connect the nodes to the physical network ph-eth1 that is created by the Rackspace cookbooks.

# ovs-vsctl add-port br-eth1 eth1

Connect to the Horizon Dashboard. Point to the Eth0 IP address of the Controller Node. You should see the Horizon Dashboard. The default administrator account is:

Username: admin
Password: secrete

Run chef-client on the nodes

Check the configuration of the nodes:


# knife node show setup1-controller
Node Name:   setup1-controller
Environment: grizzly-quantum-1
FQDN:        setup1-controller
IP:          10.112.252.244
Run List:    role[single-controller]
Roles:
Recipes:
Platform:    ubuntu 12.04
Tags:


# knife node show setup1-network
Node Name:   setup1-network
Environment: grizzly-quantum-1
FQDN:
IP:          10.112.252.245
Run List:    role[single-network-node]
Roles:
Recipes:
Platform:    ubuntu 12.04
Tags:


# knife node show setup1-compute
Node Name:   setup1-compute
Environment: grizzly-quantum-1
FQDN:
IP:          10.112.252.246
Run List:    role[single-compute]
Roles:
Recipes:
Platform:    ubuntu 12.04
Tags:


Now we are all ready to run "chef-client" command on the nodes. Run the "chef-client" command on the controller, network and compute nodes one-by-one in that order.

Add the roles to the Chef nodes runlist and assign the nodes to the environment

Use the "knife node run_list add" command to add the role to each of the Chef nodes:

# knife node run_list add setup1-controller 'role[single-controller]'
# knife node run_list add setup1-network 'role[single-network-node]'
# knife node run_list add setup1-compute 'role[single-compute]'


Use the "knife node show" command to see the node configuration:

# knife node show setup1-controller
Node Name:   setup1-controller
Environment: _default
FQDN:        setup1-controller
IP:          10.112.252.244
Run List:    role[single-controller]
Roles:
Recipes:
Platform:    ubuntu 12.04
Tags:


We see that the Environment is "_default". We need to change it to "grizzly-quantum-1" environment we created before.

Use the "knife node edit" command to edit the node configuration and change the environment from "_default" to "grizzly-quantum-1".

The node configuration will be in environment "_default" as here:

{
  "name": "setup1-controller",
  "chef_environment": "_default",
  "normal": {
    "tags": [
    ]
  },
  "run_list": [
    "role[single-controller]"
  ]
}


We need to replace "_default" with "grizzly-quantum-1". As here:

{
  "name": "setup1-controller",
  "chef_environment": "grizzly-quantum-1",
  "normal": {
    "tags": [
    ]
  },
  "run_list": [
    "role[single-controller]"
  ]
}


The same needs to be done for the other two nodes: setup1-network and setup1-compute.

Setup the Chef Environment

Under chef-cookbooks/environments directory setup the environment json file. For example create a grizzly-quantum-1.json file with the following contents:

{
  "name": "grizzly-quantum-1",
  "description": "Chef-server Grizzly Quantum Multinode environment",
  "json_class": "Chef::Environment",
  "chef_type": "environment",
  "override_attributes": {
    "nova": {
      "libvirt": { "virt_type": "qemu" },
      "network": {
        "provider": "quantum"
      }
    },
    "mysql": {
      "allow_remote_root": true,
      "root_network_acl": "%",
      "server_root_password": "fr3sca",
      "server_debian_password": "fr3sca"
    },
    "osops_networks": {
      "nova": "10.112.252.0/24",
      "public": "10.112.252.0/24",
      "management": "10.112.252.0/24"
    }
  }
}


You need to mention the Network you are using on your eth0 as the "nova", "public" and "management" attributes. Change these attributes to the network address you are using:

"osops_networks": {
"nova": "10.112.252.0/24",
"public": "10.112.252.0/24",
"management": "10.112.252.0/24"
}


Upload the environment to the Chef Server as below:

# knife environment from file grizzly-quantum-1.json


You can verify that the environment file has been uploaded to Chef Server using the following command:

# knife environment list
_default
grizzly-quantum-1

Nodes networking setup and Chef bootstrapping

Nodes Networking Setup

Use Ubuntu 12.04 LTS 64 bit.

Setup the nodes networking such that the eth0 of Controller, Compute and Network nodes are all on the same network as the Chef Workstation and this network is able to reach the Hosted Chef Server across the Internet.

The out-of-band eth0 management interfaces are where the primary IP address of the node is located, and is not controlled by OpenStack Networking.

Important: We are using Quantum networking. This is specified in the Chef environment file by using "network": { "provider": "quantum"}. The Rackspace Private Cloud cookbooks will use VLAN based networking by default. So, all eth1 interfaces should be connected to a switch using Trunk Ports. Multiple VLANs will be used to segregate the private networks that will be created using Quantum.

The eth1 physical provider interfaces have no IP addresses and must be configured to be "up" on boot. For example, to accomplish this in Ubuntu, the following configuration would need to be added to/etc/network/interfaces:

auto eth1
iface eth1 inet manual
  up ip link set $IFACE up
  down ip link set $IFACE down


The /etc/network/interfaces of controller node is as below:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface

auto eth0
iface eth0 inet static
address 10.112.252.244
netmask 255.255.255.0
gateway 10.112.252.253
dns-nameservers 10.112.116.138 10.112.116.139 10.112.64.1

auto eth1
iface eth1 inet manual
  up ip link set $IFACE up
  down ip link set $IFACE down

Bootstrapping the nodes to Chef Server

On the Chef Workstation we can run the following knife bootstrap commands to bootstrap the nodes to the Chef Server.

# knife bootstrap 10.112.252.244 -x root -P password -N "setup1-controller"
# knife bootstrap 10.112.252.245 -x root -P password -N "setup1-network"
# knife bootstrap 10.112.252.246 -x root -P password -N "setup1-compute"


Verify that the nodes are bootstrapped using 'knife node list' command:

# knife node list
setup1-compute
setup1-controller
setup1-network

Setting up your Chef Environment with Rackspace Private Cloud cookbooks

Step 1: Log into your Chef workstation

Step 2: Use git clone to download the Rackspace Private Cloud cookbooks

# git clone --recursive https://github.com/rcbops/chef-cookbooks.git

Step 3: cd into the chef-cookbooks directory

Step 4: Checkout the grizzly version of the cookbooks

# git checkout grizzly

# git submodule init
# git submodule sync
# git submodule update

Step 5: Upload the cookbooks to the Chef Server

# knife cookbook upload -a -o cookbooks
Step 6: Apply the updated roles
# knife role from file roles/*rb

Your chef cookbooks are now up to date.

Setting up the Chef Server

We can use the Opscode Hosted Chef Server itself.

Sign-up for Hosted Chef

Setup the Chef Server account on Opscode Hosted Chef Server. Go To https://manage.opscode.com and Sign up.

Save your Opscode User Private Key

Opscode User Private Key will be shown to you once you finish the signup. Save the key in ~/.chef/YOUR-USERNAME.pem

Create an Organization

Once you are done with the above steps, go to https://manage.opscode.com again and then login with the user account you just created. Then you will have to create an Organization first. Organization short name should be globally unique.

Provide the Organization name and select the free plan.

Download the Validation Key and Knife config

Copy the Validation Key and Knife config to your .chef directory

Verify Knife using the "knife --version" and "knife client list" commands.

We are done with setting up the Chef Workstation and the Chef Server.

Setting up your Chef Workstation

Install Chef

• use Ubuntu 12.04 LTS 64 bit
• wget http://opscode.com/chef/install.sh
• run the install.sh script

 

 

Introduction to Chef

Chef Server: The server acts as a hub for all of the data needed by the chef-client while it configures a node.

 

Chef Workstation: A workstation is a computer that is configured to run Knife, to synchronize with the chef-repo, and interact with a single server. The workstation is the location from which most users will do most of their work.

 

Chef Nodes: A node is any physical, virtual, or cloud machine that is configured to be maintained by a chef-client.

 

Goal: Installing Openstack using Chef