Posts Tagged ‘VMware’

All about the path

October 20th, 2011 Comments off
Reading Time: 7 minutes

Recently on the twittersphere there was a short exchange  as to why would  a security professional care about VXLAN when they don’t care about ASICs in a switch.

In stead of putting the conversation up (I’m lazy and can’t be bothered screen capturing it) I thought I’d share my $0.10 worth here.

In short, you always need to be concerned about the path of data. Just like Network Architects want to know packet paths for engineering purposes a Security Professional also is concerned with what systems or processes does it cross and where are the enforcement (choke) points. But most importantly, you need to know the data path so you can secure it.

I’ll start by briefly explaining what VXLAN is, its deficiencies and then a quick look as to why a Security professional needs to be concerned with the data paths, irrespective of whether or not they are virtual or physical.

What is VXLAN?

VXLAN (IETF REF: is “A Framework for Overlaying Virtualised Layer 2 Networks over Layer 3 Networks“. This is a VMware and Cisco (amongst others) initiative. That’s the fluffy way of saying it is a tunneling protocol, more precisely a L4 tunneling protocol.

Why have VXLAN?

Personally I wouldn’t. It is a VMware kludge to a very specific problem. However, if you are a VMware proponent then you have your uses. For any one dealing with large virtualised environments, especially multi-tennant ones, you will find that you run into the limitations of your L2 network (even a well designed one) really quickly; 4000 VLANs doesn’t go far in multi-tennant environments .

  • How do you migrate VMs across L3 boundaries?
  • How do you move devices about without changing IP addresses?
  • Do all application support the readdressing of VMs?

Let’s take a fairly simple example of vMotion; this is not the ideal example but sufficient to get the concept across. vMotion is the movement of one Virtual Machine (VM) from a physical server to another physical server. Figure 1, below, shows a fairly simple setup with servers connected to the same switch, which we will assume are in the same Layer 2 Domain.

Figure 1 – Servers with VMs in a single L2 Domain

I won’t go into the gratuitous detail on how vMotion works, you can look that up for yourself, but for the VM to move from one physical machine to another there needs to be Layer 2 (L2) adjacency. Figure 2 below shows how the machine is replicated. The path between the virtual Machine on the left (highlighted in Green); through the Hypervisor (in this instance I’ve used VMware) out the physical NIC into the switch; out the switch; into the first NIC of the second server; up through it’s Hypervisor and finally coming to rest.

Stay with me there is a point to all of this.. I hope.

Figure 2 – VMotion path through switch.

Now, throw a router in the middle (see figure 3) and the whole process breaks. Because there isn’t the ability for L2 adjacency, it crosses the L3 boundary, the machine cannot move, let alone retain its IP address, etc.

Figure 3 – VM Servers in two L2 domains, separated by router.


As mentioned previously VXLAN is essentially a tunneling protocol. It is taking a Layer 2 frames and wrapping it in a Layer 4 Datagram, Figure 4 depicts 2 VXLAN networks, green and pink. This, with some other wizardry gets over some of the aforementioned limitations, the virtual machines are tied to a VXLAN ID by the Hypervisor (they need to be the same) which is then passed to the VTEP (VXLAN Tunnel End Point) which then wraps everything in a UDP packet (OK that is slightly simplified).

Figure 4 – VXLAN allows pseudo L2 connectivity across L3.

Now there is an L2 adjacency between the two servers, they can now replicate VMs between each other whilst maintaining the same L2 and L3 addresses. See Figure 5.

Figure 5 – vMotion across L3 boundaries is now possible with VXLAN.

Security Issues:

This is where I hope the rant has some value – If you didn’t care how the protocol works (even in a rudimentary sense) or how the data flowed in the scenarios above you wouldn’t understand what the limitations, and therefore potential security flaws, are or understand what issues it solves. That said VXLAN doesn’t solve any of the existing security issues we find in Ethernet today. It still requires processes to lock down deficiencies in the technology.

  1. Just like Ethernet you can spoofing ARPs;
  2. Broadcast storms still possible (turned into multicast within VXLAN domain);
  3. No security mechanisms built into VXLAN;
  4. If you have access to the network you can spoof anything? If you are on the same network as a VXLAN segment; and
  5. Through the very nature of the VXLAN function you are also now removed from the ability to provide hardware handoff for L2-L7 security measures (think ACLs, VACLs, Firewalls, NAT, Load-balancing, etc). You need to have this gated between VXLAN domains via a VM that spans both.

Even ignoring all these new issues (yes they are new issues, is this a new way of accessing an environment, of course it is!) what about policies, enforcement points, network taps, etc, that may be already in place, in-line/path with the new VXLAN tunnel that you are proposing?

These are all why a security engineer/architect/designer needs to care about the path.

Now to the comment “security guys don’t care about the path through a switch’s ASIC…”, generally speaking, most won’t care, but they should. Just because it is a switch doesn’t mean that nothing happens to a packet once it goes in and then goes out the other side. ASICs are customised, programmable chips that allow the device to perform a number of different functions; if you’ve ever looked into this you will know that no two platforms are the same (even from the same vendor).

In the diagram below (which is a fictitious switch construct) you can see that there are some capabilities built into the different components of the switch, specifically the ASICs.

Getting traffic from one port to another port isn’t necessarily a simple thing. Can I get straight from Eth1 to Eth2 directly, or do I have to go up through the switching fabric to the L2 switching engine? What happens when I have to route (below shows a path of a packet when going via the routing engine)? Does, or can, the packet be modified/duplicated inside the switch and if so what happens? At what point in the packet’s path through the switch is policy enforced, and in what order?

The list goes on and on.

Most enterprise switches, even basic ones, will allow you to mirror ports, I’m pretty sure that Security folk would care about what ports are set up as mirrors and more importantly, what’s at the other end of that port.


The scenario’s above are fairly simple, more advanced switches allow far more sophisticated things to happen. You can rewrite packets, pass off packets to other processors (Routing, Load balancing, Packet inspection, policy enforcement, etc). Again you need to know the path through the switch and what touches a packet to understand the risk and potential impact to that application.


Chris Neal sums up my rant nicely below:

Inter-cloud interoperability

August 18th, 2009 Comments off
Reading Time: 2 minutes

VMware_ESXiWith all the work going into things like OVF and working out how we’re going to make sure that the applications/servers operate in the way we want across supplier there is a few critical things that have yet to be addressed… These are Network related, specifically, Network portability, QoS and Security zone transference.

Currently OVF supports information around the virtual system i.e. Virtual Hardware Selection which could include a number of different components including Ethernet adapters, system requirements and a range of other options. Currently there isn’t a way to specify more detail.

For example the Ethernet Adapter section only allows for Logical Name grouping (OK and MAC address specification, but I won’t go there just yet), where all VMs with the same Ethernet Logical name are placed in the same segment. That’s well and good but how does that translate from my private, or corporate, network to a hosted one? Currently it doesn’t.

Whilst there is nothing on QoS, it is a very complex piece of negotiating ensuring that something gets the correct network priority (also only really applicable when looking at private links between hosting company and customer) when dealing inter-company (Then again priority of 1 VM over another in your sub domain, is a very real thing).

How about security zoning? Surely there could be a simple untrusted, semi-trusted and trusted/secure model put forward, ensuring that applications bundled in the OVF framework could be virtually segmented and separated by firewalls or just basic ACLs? I’m not saying that the rule base needs to be passed on, seriously, when have application developers been that switched on that they unerstand what port is on what, etc? But it would allow for smarter integration with something like Chris Hoff’s A6 idea. and ease some of the regulatory compliance pain.

I’ll be watching the standard closely as it progresses and more so once VMware’s vCloud platform is officially launched.

VMware – What is Cloud Computing?

August 11th, 2009 Comments off
Reading Time: 1 minutes

krusty.jpegWell, if you saw the Cisco, VMware and EMC presentation I went to yesterday, the view  of Mike DiPetrillo from VMware is something like

an elastic/dynamic and interoperable environment.

That is, it is only “cloud” if it is fully flexible, fully scalable and fully portable.. Oh, and only if it “runs on VMware” that is.

The main catch cry was “don’t care”. The closer you get to Don’t care the closer to cloud you are. I struggled with that as I am a Network Engineer first, security bigot second and at the end of the day I’m the poor monkey that needs to build this stuff. However I can see where it was going, the better the abstraction, flexibility and freedom the less organisations need to care HOW it’s done, and can worry more about SLAs and focusing on their core business.

Whilst I understood and agreed with what Mike had to say, I still believe it’s a limited view.

UPDATE: VMware announced acquisition of Spring Source.. Will be interesting to see where this takes them.