The Technology Evangelist – Exploring the leading edge of computing ...

The Technology Evangelist (https://technologyevangelist.co/) Exploring the leading edge of computing technology with a focus on networking Search …

What the FEC? Auto-Detect is Finally Here for 25GE! (https://technologyevangelist.co/2018/02/ 18/25g-what-the-fec/)

Search

Articles by Email [email protected] Subscribe

by scottcschweitzer on February 18, 2018

As technology marches forward new challenges arise that were not previously an issue. Consider as mankind moved from walking to horseback we cleared trails where there was once brush covered paths. As we transitioned from horseback to carriages those paths needed to become dirt roads, and the carriages added suspension systems. With the move from carriages to automobiles, we further smoothed the surface traveled by adding gravel. As the automobiles moved faster, we added an adhesive to the gravel creating paved roads. With the introduction of highways, we required engineered roads with multilayered surfaces. Each generation reduced the variability in the road surface by utilizing new techniques that enabled greater speed and performance. The same holds true for computer networks. Over the past three decades as we transitioned from 10Mbps to 25Gbps Ethernet we’ve required many innovations to support these greater speeds. The latest of these being Forward Error Correction (FEC). The intent of FEC is to reduce the bit error rate (BER) as the cable length increases. In 2017 we saw the ratification of the IEEE 25GbE specification which provides two unique methods of FEC. There is BASE-R FEC (also known as Firecode) and RS-FEC (known also as Reed Solomon). Both of these FEC algorithms introduce additional network latency as the

Top 5 in Past Day What's a Smart NIC? (https://technologyevangelist.c o/2017/07/17/whats-a-smartnic/) What the FEC? Auto-Detect is Finally Here for 25GE! (https://technologyevangelist.c o/2018/02/18/25g-what-thefec/) Kernel Bypass = Security Bypass (https://technologyevangelist.c o/2017/12/05/kernel-bypasssecurity-bypass/) A 10GbE Capture Platform: Snort, Bro, Suricata & Wireshark (https://technologyevangelist.c o/2015/03/18/a-10gbe-captureplatform-snort-bro-suricatawireshark/) The New Network Edge - The Server (https://technologyevangelist.c o/2018/02/10/the-newnetwork-edge-the-server/)

signal is decoded, BASE-R is about 80 nanoseconds while RS-FEC is about 250 nanoseconds. The complexities don’t end here though, it turns out there are three different Direct Attach (DA) cable types with varying levels of quality, from good, to best we have: CA-25G-L: up to 5m, requires RS-FEC CA-25G-S: up to 3m, lower loss, requires either RS-FEC or BASE-R FEC CA-25G-N: up to 3m, even lower loss, can work with RS-FEC, BASE-R FEC, or no FEC But wait there’s more, if you order now we’ll throw in auto-negotiation (AN) and link training (LT) as both are required by the 25GbE IEEE standard (10GbE didn’t need these tricks). So what does AN actually negotiate? Two things, link speed and which type, if any, FEC will be utilized. It should be noted that existing 25GbE NICs that have been on the market likely only support one type of FEC. As for LT, it helps to improve the quality of the 25GbE link itself. It turns out though that the current generation of 25GbE switches came out prior to AN being worked out so support is at best poor to mixed. Often manual switch and adapter configuration are required. Oh, and did I mention that optical modules don’t support AN/LT? Well, they don’t, but some will support short links with no FEC. So where does this leave people who want to deploy 25GbE? You need to be careful that both your network switch and server NICs will work well together. We strongly advise that you do a proof of concept prior to a full deployment. Not all 25G server NICs do both AN/LT because their chips (ASICs) were designed and fabricated prior to the completion of the IEEE specification for 25GbE last year.

Recent Podcasts TE18: Personal Cyber Security (https://technologyevangelist.c o/podcast/te-18-personalcyber-security/) March 1, 2018 TE17: The Low Down on The Meltdown (https://technologyevangelist.c o/podcast/te17-the-low-downon-the-meltdown/) January 18, 2018 TE16: Digital Currencies & Bitcoin – Full Discussion (https://technologyevangelist.c o/podcast/te16-digitalcurrencies-bitcoin-fulldiscussion/) December 11, 2017 TE15: Bitcoin, Part 7 of 7 Digital Currencies (https://technologyevangelist.c o/podcast/te15-bitcoin-part-7of-7-digital-currencies/) December 11, 2017 TE14: Bitcoin, Part 6 of 7 Digital Currencies (https://technologyevangelist.c o/podcast/te14-bitcoin-part-6of-7-digital-currencies/) December 6, 2017

Solarflare’s 25GbE X2522 server NICs which debut next month include support for all the above, in fact, when initially powered up they will begin by: First looking at cable, is it SFP or SFP28?

(https://itunes.apple.com/us/po dcast/the-technology-

If it’s SFP28 it will attempt AN/LT, then 25G no AN/LT, then 10G

evangelist/id1253336133?

If it’s a 25G link, then it will try and detect which FEC is being used by switch

mt=2) (https://playmusic.app.goo.gl/?

Additionally, the server administrator can manually override the defaults and select

ibi=com.google.PlayMusic&isi

AN/LT and the FEC type and setting (auto, on, off).

=691797987&ius=googleplay

I grew up in New York, and remember listening to Sy Sims on TV say “an educated consumer is our best customer…”

music&link=https://play.googl e.com/music/m/I5hlryarb6wn5 zzoddfgpoynuna? t%3DThe_Technology_Evange list%26pcampaignid%3DMKT

P.S. I’d like to give a special thanks to Martin Porter, Solarflare’s VP of

-na-all-co-pr-mu-pod-16)

Engineering, for pulling all this together into a few slides. { Add a Comment (https://technologyevangelist.co/2018/02/18/25g-what-thefec/#respond) }

Visibility + Control = Orchestration (https://technologyevangelist.co/2018/02/ 12/visibility-control-orchestration/) by scottcschweitzer on February 12, 2018

In Taekwondo to win you watch your opponent’s center of gravity (CoG), for the eyes lie. For example, if the CoG moves toward their back foot you can expect a front kick, or if it begins a slight twist without moving forward or backward then a punch from the arm in the direction of the twist is coming. These are mandatory anticipatory movements which are precursors to a pending threat. If my opponent throws a punch or launches a kick without these movements it will be ineffectual. A punch without a twist is a tap. Of course the above is no secret. Skilled attackers lead with a feint to disguise their real intent, but that’s for another time. Cybersecurity is no different, you need to detect a threat, see it, classify it, then act on it. Detecting and seeing the threat is commonly referred to as Visibility. Classifying then acting on the threat is called Orchestration. Imagine if you could watch the CoG of every server in your data center? In cyber terms that CoG might be every data flow in/out of the server. Placing boundaries and alerts on those flows is the primary role of orchestration. Placing these boundaries is now called micro-segmentation. Recently we suggested that the New Network Edge is the server itself. Imagine if you could watch every data flow from every server, set up zero trust policies to govern in advance which flows are permitted, then the system generates alerts to security operations when other flows are attempted. With solid governance comes the capability to quarantine applications or systems that have gone rogue. All the while all of this is done within the server’s own NICs, without any host agents or utilizing any local x86 CPU cycles, that’s Solarflare ServerLock. Below is a screenshot of ServerLock displaying seven groups of hosts, in the dark grey bubbles, with all the flows between those hosts in red. The Database servers group is highlighted, and all the network flows for this group are shown. Note this is a demonstration network. Click on the image below to see a larger version of it.

(https://i2.wp.com/technologyevangelist.co/wpcontent/uploads/2018/02/ui_demo1.png?ssl=1) { Add a Comment (https://technologyevangelist.co/2018/02/12/visibility-controlorchestration/#respond) }

The New Network Edge – The Server (https://technologyevangelist.co/2018/02/ 10/the-new-network-edge-the-server/) by scottcschweitzer on February 10, 2018

Today cleverly crafted spear phishing emails and drive-by downloads make it almost trivial for a determined attacker to infect a corporate workstation or laptop. Wombat’s “State of the Phish 2018 (https://www.wombatsecurity.com/hubfs/2018%20State%20of%20the%20Phish/Wo mbat-StateofPhish2018.pdf?submissionGuid=87d041b0-37da-4eb0-ac82d765a68b3db2)” report shows that 76% of InfoSec professionals experienced phishing attacks in 2017. Malware Remote Access Toolkits (RATs) like Remcos (https://breaking-security.net/remcos/) for Windows can easily be rebuilt with a new name and bound to legitimate applications, documents or presentations. Apple Mac users, myself included, are typically a smug group when it comes to Malware so for them, there’s MacSpy (https://www.scmagazine.com/researchers-spot-macmalware-as-a-service-dubbed-macspy/article/667978/) which is nearly as feature rich. A good RAT assumes total control over the workstation or server on which they are installed then it leverages a secure HTTPS connection back to their command and control server. Furthermore, they employ their own proprietary encryption techniques to secure their traffic prior to HTTPS being applied. This prevents commercial outbound web proxies designed to inspect HTTPS traffic from gaining any useful insights into the toolkits nefarious activities. With the existence of sophisticated RATs, we must reconsider our view of the enterprise network. Once a laptop or workstation on the corporate network is compromised in the above fashion all the classic network defenses, firewalls, IDS, and IPS are rendered useless. These toolkits force us to reconsider that the New Network Edge is the server itself, and that requires a new layer in our Defense in Depth model. The data on our enterprise servers are the jewels that attackers are paid a hefty sum to acquire. Whether it’s a lone hacker for hire by a competitor, a hacktivist group or a rogue nation state, there are bad actors looking to obtain your companies secrets. Normally the ONLY defenses on the corporate network between workstations and servers are the network switches and software firewalls that exist on both ends. The network switches enforce sub-networks (subnets) and virtualized local area networks (VLANs) that impose a logical structure on the physical network. Access Control Lists (ACLs) then define how traffic is routed across these logical boundaries. These ACLs are driven by the needs of the business and meant to reflect how information should flow between different parts of the enterprise. By contrast, the software firewalls on both the workstations and servers also define what is permitted to enter and leave these systems. As defenses, both these methods fall woefully short, but today they’re the last line of defense. We need something far more rigorous that can be centrally managed to defend the New Network Edge, our servers. As a representation of the businesses processes, switch ACLs are often fairly loose when permitting systems on one network access to those on another. For example, someone on the inside sales team sitting in their cubical on their workstation has access to the Customer Relationship Management (CRM) system which resides on a server that is physically somewhere else. The workstation and server are very likely on different subnets or VLAN within the same enterprise, but ACLs exist that enable the sales person’s workstation access to customer data provided by the CRM system. Furthermore, that CRM system is actually pulling the customer data from a third system, a database server. It is possible that the CRM server and the database server may be on the same physical server, or perhaps in the same server rack, but very possibly on the same logical network. The question is, is there a logical path from the inside sales person’s workstation to the database server, the answer should be no, but guess what? It doesn’t matter. Once the inside salesperson is successfully spear fished then it’s only a matter of time before the attacker has access to the database server through the CRM server. The attacker will first enable the keylogger, then watch the sales person’s screen to see what they are doing, harvest all their user ids and passwords, perhaps turn on the microphone and listen to their conversations, and inspect all the outgoing network connections. Next, the attacker will use what they’ve harvested and learned to begin their assault, first on the CRM server. Their goal at this point is to establish a secondary beachhead with the greatest potential reach from which to launch their primary assault while keeping the inside sales person’s workstation as their fallback position. From the CRM server, they should be able to easily access many of the generic service machines: DNS, DHCP, NTP, print, file, and database systems. The point here is that where external attackers often have to actively probe a network to see how it responds, internal RAT based attacks can passively watch and enumerate all the ports and addresses typically used. In doing so they avoid any internal dark space honeypots, tripwires, or sweep detectors. So how do we protect the New Network Edge, the server itself? A new layer needs to be added to our defense in depth model called microsegmentation or application segmentation. This enforces a strict set of policies on the boundary layer between the server and the network. Cisco, Arista, and other switch providers, with a switch-based view of the world, would have you believe that doing it in the switch is the best idea. VMWare, with its hypervisor view of the world, would have you believe that their new NSX product is the solution. Others like Illumio and Tuffin would have you believe that a server-based agent is the silver bullet for micro-segmentation. Then there’s Solarflare, a NIC company, with its NIC based view of the world, and its new entrant in the market called ServerLock. Cisco sells a product called Tetration (https://www.cisco.com/c/en/us/products/datacenter-analytics/tetration-analytics/index.html) designed to orchestrate all the switches within your enterprise and provide finely grained micro-segmentation of your network traffic. It requires additional Cisco servers be installed to receive traffic flow data from all the switches, processes the data, then provides network admins with both the visibility and orchestration of the security policies across all the switches. There are several downsides to this approach, it is complex, expensive, and can very possibly be limited by the ACL storage capabilities of the top of rack switches. As we scale to 100s of VMs per system or 1,000s of containers these ACLs will likely be stretched beyond their limits. VMWare NSX includes both an advanced virtual switch and a firewall that both require host CPU cycles to operate. Again, as we scale to 100s of VMs per system the CPU demands placed on the system by both the virtual switch and the NSX firewall (https://blogs.vmware.com/networkvirtualization/2015/10/getting-startedwith-vmware-nsx-distributed-firewall-part-2.html/) will become significant, and measurable. Also, it should be noted that being an entirely software-based solution NSX has a large attackable surface area that could eventually be compromised. Especially given the Meltdown and Spectre vulnerabilities recently reported by Intel. Finally, VMWare NSX is a commercial product with a premium price tag. This brings us to the agent-based solutions like Illumio (https://www.illumio.com/solution-architecture) and Tuffin (https://www.tufin.com/tufin-orchestration-suite). We’ll focus on Illumio which comes with two components the Policy Compute Engine (PCE) and the Virtual Enforcement Node (VEN). The marketing literature states that the VEN is attached to a workload, but it’s an agent installed on every server under Illumio’s control and it reports network traffic flow data into the PCE while also controlling the local OS software firewall. The PCE then provides visualization and a platform for orchestrating security policies. The Achilles heel of the VEN is that it’s a software agent which means that it both consumes x86 CPU cycles and provides a large attackable surface area. Large in the sense that both its agent and the OS-based firewall on which it depends can both be easily circumvented. An attacker need only escalate their privileges to root/admin to hamstring the OS firewall or disable or blind the VEN. Like VMWare NSX, Illumio and Tuffin are premium products. Finally, we have Solarflare’s NIC based solution called ServerLock (http://solarflare.com/solarsecure). Unlike NSX and Illumio which rely on Intel CPU cycles to handle firewall filtering, Solarflare executes its packet filtering engine entirely within the chip on the NIC. This means that when an inbound network packet is denied access and dropped it takes zero host CPU cycles, compared to the 15K plus x86 cycles required by software firewalls like NSX or IPTables. ServerLock NICs also establish a TLS-based domain of trust with a central ServerLock Manager similar to Illumio’s PCE. The ServerLock Manager receives flow data from all the ServerLock NICs under management and provides Visibility, Alerting and Policy Management. Unlike Illumio though the flow data coming from the ServerLock NICs requires no host CPU cycles to gather and transmit, these tasks are done entirely within the NIC. Furthermore, once the Solarflare NIC is bound to a ServerLock Manager the local control plane for viewing and managing the NIC’s hardware filter table is torn down so even if an application were to obtain root privilege there is no physical path to view or manage the filter table. At this point the, it is only capable of being changed from the specific ServerLock Manager to which it is bound. All of the above comes standard with new Solarflare X2 based NICs that are priced at or below competitive Intel NIC price points. ServerLock itself is enabled as an annual service sold as a site license. So when you think of micro-segmentation would you rather it be done in hardware or software? P.S. Someone asked why there is a link to a specific RAT or why I’ve included a link to an article about them, simple it validates that these toolkits are in-fact real, and readily accessible. For some people, threats aren’t real until they can actually see them. Also, another person asked, what if we’re using Salesforce.com, that’s ok, as an attacker instead of hitting the CRM server I’ll try the file servers, intranet websites, print servers, or whatever that inside salesperson has access to. Eventually, if I’m determined and the bounty is high enough, I’ll have access to everything. { Add a Comment (https://technologyevangelist.co/2018/02/10/the-new-networkedge-the-server/#respond) }

Onload Recovers Meltdown Lost Performance (https://technologyevangelist.co/2018/01/ 08/onload-recovers-performance-lost-tomeltdown-patch/) by scottcschweitzer on January 8, 2018

The recently announced microprocessor architecture vulnerability known as Meltdown (https://meltdownattack.com/meltdown.pdf) is focused on accessing memory that shouldn’t be available to the currently running program. Meltdown exploits a condition where the processor allows an unprivileged application the capability to continually harvest data unrestricted from anywhere in system memory. The flaw that enables Meltdown is based on microprocessor performance enhancements more than a decade old and are now common in Intel and some ARM processors. The solution to Meltdown is Kernel page-table isolation (https://en.wikipedia.org/wiki/Kernel_pagetable_isolation) (KPTI), but it doesn’t come without a performance impact which ranges from 5-30% (https://www.phoronix.com/scan.php?page=article&item=linux415-x86pti&num=1), every application behaves differently. Since Onload places the communications stack into that application’s userspace this dramatically reduces the number of kernel calls for network operations and as such avoids most of the performance impact brought on by KPTI. Redhat confirm this in a recent article (https://access.redhat.com/articles/3307751#comments) on this topic. This means that applications leveraging Onload on KPTI patched kernels will see an even greater performance advantage. By contrast Spectre (https://spectreattack.com/spectre.pdf) tears down the isolation that exists between running applications. It allows a malicious application to trick error-free programs into leaking their secrets. It does this by scanning the process address space of those programs, and the kernel libraries on which they depend, looking for exploitable code. When this vulnerable code is executed it acts as a covert channel transmitting its secrets to the malicious application. This vulnerability affects a wider range of processors and requires both kernel and CPU microcode patches, and even then, the vulnerability hasn’t been 100% eliminated. More work remains to be done to shut down Spectre. { Add a Comment (https://technologyevangelist.co/2018/01/08/onload-recoversperformance-lost-to-meltdown-patch/#respond) }

Post Spectre/Meltdown, Are We More Secure? (https://technologyevangelist.co/2018/01/ 04/post-specter-meltdown-are-we-moresecure/) by scottcschweitzer on January 4, 2018

It’s human nature in time-critical environments to speculate and execute, remember Radar O’Reilly from M.A.S.H., it’s how we operate at our best. Sometimes you’re wrong and you throw out that work, but the majority of the time you’re right, and the payoff, in the case of an army field hospital could be life-saving. For computer processors, up until recently, that payoff was less critical and the savings could be measured in nanoseconds per successful speculation. Now while nanoseconds on their own don’t sound like much, consider that this savings comes with EVERY successful speculation. Also, it should be noted that there isn’t one block on the processor that is speculating and executing, but rather dozens of them in parallel. Current projections are that patching around the “speculate and execute” set of three flaws could reduce system performance from 5-30% (https://en.wikipedia.org/wiki/Kernel_page-table_isolation). Friday Apple announced (https://www.engadget.com/2018/01/04/apple-spectremeltdown-intel-arm-cpu-bug/) that they’ve patched all their OSes to mitigate just the Meltdown (https://meltdownattack.com/meltdown.pdf) flaw and at worst the performance hit was only 2.5%, they are still working on Spectre. Redhat (https://www.redhat.com/en/blog/what-are-meltdown-and-spectre-here’s-what-youneed-know?source=onsite&key=specter%20meltdown), which has worked closely with Google’s Project Zero (https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-withside.html), has not only addressed Meltdown but also Spectre (https://spectreattack.com/spectre.pdf) by releasing patches for both sets of flaws. Many articles and blog posts this week will cover the details of Spectre and Meltdown (https://arstechnica.com/gadgets/2018/01/meltdown-and-spectre-everymodern-processor-has-unfixable-security-flaws/), and as such, this one won’t. Also, they will devote time on the lengths to which Intel, AMD, ARM, Linux, Apple, Microsoft, Google, Amazon, and others will or have gone to plug these holes, this post won’t. The real underlying question is are we more or less safe today as a result of these discoveries? My view is that this is Shellshock (https://en.wikipedia.org/wiki/Shellshock_(software_bug)) only three years later and in silicon. Let’s flashback to 2014 when Stephane Chazelas discovered a critical security flaw in the Bash Shell after it had been in production for several decades. Within several days five more security flaws were found in the same code. Code that was widely considered stable and trusted. Clearly, it had never been revisited and retested using state-of-the-art hacking techniques. The same was true with speculative execution (https://en.wikipedia.org/wiki/Speculative_execution). It has been a microprocessor design component since the early days of pipelining, possibly as far back as the original IBM ROMP (https://en.wikipedia.org/wiki/ROMP) processor of 1981 (precursor to RISC and later the Power Architecture). It appears that chip security experts are now revisiting old trusted silicon techniques with new eyes looking for potential current exploits. This can only be a good thing as it will further secure the computing platforms on which we rely. In the near term, patching for Spectre (https://spectreattack.com/spectre.pdf) and Meltdown (https://meltdownattack.com/meltdown.pdf) will improve security at the expense of performance, but it will only take another chip spin to likely recover that lost ground. At which point we’ll have safer and more trustworthy platforms. While originally published on January 4th, 2018, this piece has since been updated on January 6, 2018. { Add a Comment (https://technologyevangelist.co/2018/01/04/post-spectermeltdown-are-we-more-secure/#respond) }

2600, Attacking Enterprise Networks (https://technologyevangelist.co/2018/01/ 02/2600-attacking-enterprise-networks/) by scottcschweitzer on January 2, 2018

Since 1984 the magazine “2600 (https://store.2600.com)” has been the undisputed publication created by and for hackers and phone phreakers worldwide. The January 2017 cover to the right is misleading, the magazine isn’t named after the Atari system, but rather the 2600 Hz tone (https://en.wikipedia.org/wiki/2600_hertz) ATT used, that phreakers leveraged, to control long distance phone lines. Most article bylines are hacker handles, rather than proper names, as the articles themselves don’t always paint inside the numbers. In January 2017 a hacker with the handle “Daelphinux” published the first of a five-part series of articles titled “Successful Network Attacks – Phase One (https://www.amazon.com/2600-Magazine-HackerQuarterly-2016-2017-ebook/dp/B01N7O29CQ/ref=sr_1_fkmr0_1?s=digitaltext&ie=UTF8&qid=1514739362&sr=1-1fkmr0&keywords=2600+hacker+digest+winter+2017)” and every quarter since then he’s published the next phase, with the final Phase Five hitting the streets today January 2, 2018. This collection of five articles is perhaps the most concise executive-level summary of how an attacker breaches an enterprise that I’ve read thus far. Hopefully, I won’t offend Daelphinux (https://hackaday.io/Daelphinux) by attempting to summarizing the key points of his five articles in this single blog post. I strongly suggest that everyone reading this review the original text in these five issues of 2600. Daelphinux classified a Successful Network Attack into five phases: 1. Reconnaissance – “Gathering as much useful information about the targets as possible.” 2. Scanning – “Gathering useful information about the target’s networks and any possible exploits.” 3. Gaining Access – “Getting into the network to be able to accomplish the attack’s goal.” 4. Maintaining Access – “Ensuring access to the network persists long enough to accomplish the attack’s goal.” 5. Covering Tracks – “Obfuscating the attacker’s presence on the network such that they cannot be traced.” Each phase builds on the first, with Daelphinux envisioning this as a pyramid with phase one on the bottom, and each successive phase building on the prior one. Attackers will need tools and skills at each level in order to conduct a successful attack. Defenders, the enterprise admins, will also need tools and skills for several phases to detect and defend against an attack. Phase 1 – Reconnaissance. As defined by Daelphinux the attacker seeks to gather all the raw data they can on their target before actively engaging them. The key here is in gathering only the “useful” data, as an attacker will rapidly accumulate an enormous pile of information. This information should come from a wide variety of sources including, but not limited to: deep web searches, web crawling those results, calling all the targets publicly available phone numbers to learn everything they can about the target, draining “whois” databases for all known corporate DNS assets, launching phishing and email scams at targeted employes and generic email address, real-life social engineering by making new friends, and finally dumpster diving. All these efforts will produce a heap of information, but most of it will be useless. Here is where intelligent sifting comes in. Important information is often in handwritten notes, or cast of printouts such as printer test configuration pages, IP table listings, usernames, equipment manufacturer shipping boxes, operating system manuals, internal organizational charts & structures, and corporate policies (especially password). Recognizing these reconnaissance efforts is the initial step toward thwarting an attack. For example, reviewing security footage looking for dumpster divers might sound trivial, but differentiating between an attacker and someone looking something to sell for their next meal can be challenging. Other activities like social engineering become far more complex to detect via video surveillance as these activities appear as background noise. While this phase might be the toughest to detect, it is the easiest to defend against. If you can cut off the flow of information outside enterprise you can seriously hamper their reconnaissance efforts. To do this you can hide the whois records, destroy printed copies of purchase orders, destroy shipping boxes used to pack new servers or appliances, destroy discarded manuals, remove and clean printer hard drives, make sure all in-house shredders use cross-cut shredding and finally burn any really sensitive info that has already been shredded. Much of this can be addressed through procedures and training. Phase 2 – Scanning, In April Daelphinux covered the details of this phase. He said that when an attacker moves on to phase two they are committed, and unlikely to walk away. This phase is the first real step where the attacker can’t evade detection as they have to deploy active tools to electronically gather as much insight as they can. Tools like: nmap, Nessus, Metasploit, ZAP, Xenotix or Grabber. Here they are looking to generate IP Maps, enumerate subnets, determine network speeds, the resilience of networks, open ports on clients, appliances & servers used, along with applications and the versions in production. All this scanning will provide the attacker with another huge heap of data to sift through with the eventual goal being to define the attack vectors that define the real “meat” of the attack. This phase is the last chance for a defender to stop an attack before valuable assets are stolen. During this phase, administrators might notice network slowdowns, so on detecting these and investigating see if one IP address or a small range of addresses is touching a wide range of resources if so you are likely in the process of being scanned. Attackers will often launch the scanning phase remotely. So using network address translation (NAT) internally, next-generation firewalls, and current IPS & IDS appliances can in some cases detect this. It is always strongly recommended to be current in patching all your appliances and to follow proper admin process. Phase 3 – Gaining Access, was published in 2600 in July of 2017. Without gaining access an attack isn’t an attack. Some of the key tools used are Metasploit and ZAP, but they will also leverage trojans, zombies, and backdoors to gain multiple toeholds into the enterprise’s network. Attackers often use remote shells instead of graphical tools, as shells often prove to be faster, more flexible and powerful. Typically attackers operate from an endpoint that is not their intended target, for example leveraging another user’s machine to attack a server. By using another unsuspecting endpoint if it becomes compromised they can then move onto another workstation within the enterprise network. Attackers are typically interested in copying, moving, deleting and or altering files. In this article we’re not interested in those seeking to launch a Denial of Service (DoS) attack, just those looking to exfiltrate value from the enterprise. Detecting attacks during this phase requires active monitoring, and the question is when, not if, you’ll detect an attacker. Doing active file audits, examining files that are changed outside of regularly expected intervals, watching for irregular traffic patterns, and reviewing access and error logs are all known methods of detecting an attack. Phase 4 – Maintaining Access, was published in the October 2017 issue. This is the last phase where defending against an attack can successfully prevent data loss. Daelphinux stated that there are three things which should be kept in mind: 1. The Attacker has already breached the network. 2. The Attacker is actively attempting to achieve their goal. 3. Less experienced attackers tend to get overly comfortable with their success at this point. This is the stage at which attackers are at their most vulnerable. They are moving around in plain sight within your network and can be detected if you’re looking for them. Network monitoring is the key to detection in this phase, but also, unfortunately, detection at this phase is also based on hope. You as the defender are hoping you discover them, and they are hoping their connection won’t get them detected and severed. Skilled attackers will setup precautions to prevent this. One of the attacker’s strategies will be to take over network monitoring tools in an effort to circumvent detection. Enterprises need a heavy level of paranoia at this point to ensure that they are checking everything. Looking for the use of uncommon ports or protocols is another method for detecting attackers. Typically Intrusion Prevention (IPS) and Intrusion Detection Systems (IDS) and Security Information and Event Management Systems (SIEMS) are useful tools in ferreting out hackers. SIEMS themselves often are now targets for attackers. Multiple instances of these devices or appliances should be leveraged to thwart a takedown. Consider having these monitors watch each other, so in the event, one goes down you can use that as a potential indicator of an attack. If both or all monitors go down simultaneously then it’s very possible you’re under attack. Killing connections that meet certain criteria are vital to cutting off an attackers access, for example terminating connections that last longer than 25 minutes. Think that scene in the original “Transformers” when the general shouts “Cut the hard lines!” Phase 5 – Covering Tracks, today January 2, 2018, we saw the latest issue of 2600 with Daelphinux final installment in the series on “Successful Network Attacks – Phase Five – Covering Tracks.” Once an attacker has reached phase five they’ve taken what they need, and now the coverup begins. Attempting to defend against this phase “is a form of damage control,” at best you’re preserving forensic evidence to hopefully reconstruct what they took after the fact. An attacker has to leave as cleanly as they entered otherwise they could dilute some of the value of what was taken. As Daelphinux points out, what good is a list of users and passwords if the passwords have all been changed. The same holds true of Malware and backdoors, you can’t sell a backdoor into an enterprise if it has been removed. The best defense against this is redundant, and perhaps even hidden copies of logs. Attackers will often sanitize the logs they find, but they rarely go looking for additional copies of logs, especially if some effort has been made to hide and even secure them. It’s possible that if you have automated your logging such that multiple copies are generated, and accesses are tracked the attacker will notice this in phase three and just avoid it. Attackers will normally obfuscate both their IP and MAC addresses to further frustrate tracking them, often using addresses already on the network. Again, as mentioned in phase three setting connection limits, timeouts, and alerts when these are reached is often a good way to thwart or even detect an attacker. It should also be noted that attackers will often escalate their privilege on a system so they can disable logging. As Daelphinux noted disabling logging will often generate its own log event, but then after that, you won’t know what was done. Some attackers may even just erase or corrupt log files on the way out the door, a sort of “salt the earth” strategy to make determining what was taken that much more difficult. Regardless, a company will need to make some determinations on what was stolen or affected and alert their customers. A recent Ponemon report states that just cleaning up and dealing with a data breach in the US often costs companies $244/record. Hats off to Daelphinux for authoring “Successful Network Attacks,” and “2600, The Hacker Quarterly” for publishing it. I hope everyone has a Happy and Secure New Year. { Add a Comment (https://technologyevangelist.co/2018/01/02/2600-attackingenterprise-networks/#respond) }

Idea + technology + {…} = product (https://technologyevangelist.co/2017/12/ 26/idea-technology-product/) by scottcschweitzer on December 26, 2017

If you’ve been in this industry a while you may remember the IBM Simon (https://en.wikipedia.org/wiki/IBM_Simon) or the Apple Newton

(https://en.wikipedia.org/wiki/Apple_Newton), both great ideas for products, but unfortunately, the technology just wasn’t capable of fulfilling the promise that these product designers had in mind. The holidays provide a unique opportunity to reflect. They also simultaneously create an environment for an impulse buy proceeded by a pause every year to play with my kids (now 21 and 24). 2017 was no different, and so this year for the first time ever I picked up not one but three quadcopter drones. After dinner Christmas day, all three were simultaneous buzzing around our empty two car garage attempting to take down several foam rubber cubes balanced on the garage door opener return beam. Perhaps I should bound this a bit more, a week earlier I’d spent $25 on each of the kid’s drones, not knowing if they would even interested, and $50 on my own. We had a blast, and if you’ve not flown one you should really splurge and spend $50 for something like the Kidcia (https://www.amazon.com/gp/product/B06WD4JYPL/ref=oh_aui_detailpage_o03_s 00?ie=UTF8&psc=1) unit, it’s practically indestructible. On the downside, the rechargeable lithium batteries only last about eight minutes, so I strongly suggest purchasing several extra batteries and the optional controller. During the past week since these purchases, but before flying, I’ve wondered several times why we haven’t seen life-sized quad-copter drones deployed in practical real-world applications? It turns out this problem has waited 110 years for the technology. Yes, the quadcopter or rotary wing aircraft was first conceived, designed and demonstrated, in tethered flight mode back in 1907 (https://en.wikipedia.org/wiki/Breguet-Richet_Gyroplane). The moment you fly one of today’s quadcopters you quickly realize why they flew in tethered flight mode back in 1907, crashing is often synonymous with a landing. These small drones, mine has folding arms and dual hinged propellers, take an enormous beating and still continue to fly as if nothing happened. We put at least a dozen flights on each of the three drones on Christmas day, and we’ve yet to break a single propeller. Some of the newer, more costly units, now include collision avoidance, which may actually take some of the fun away. So back to the problem at hand, why has it taken the quadcopter over 110 years to gain any traction beyond concept? Five reasons stand out, all technological, that have made this invention finally possible: Considerably computing power & sophisticated programming in a single ultralow power chip Six-axis solid-state motion sensors (3-axis gyroscope, 3-axis accelerometer) also on a single ultra-low power chip Very high precision, efficient, compact lightweight electric motors Compact highly efficient energy storage in the form of lithium batteries Extremely low mass, highly durable, yet flexible propellers That first tethered quadcopter back in 1907 achieved only two feet of altitude while flown by a single pilot and powered by a single motor with four pairs of propellers. Two of the pairs of propellers were counter-rotating to eliminate the effects of torque, and four men, aside from the pilot, were required to keep the craft steady. Clearly far too many dynamically changing variables for a single person to process. Today’s quadcopter drones have an onboard computer that continuously adjusts all four motors independently while measuring the motion of the craft in six axises and detecting changes in altitude (via another sensor). The result is that when a drone is properly setup it can be flown indoors and raised to any arbitrary altitude where it will remain hovering in place until the battery is exhausted. Once the pilot requests the drone move left to right, all four rotors speeds are independently adjusted via the onboard computer to keep the drone from rotating or losing altitude. Controlled flight of a rotary wing craft, whether a drone or a flying car, requires considerable sensor input, and enormous computational power. Petroleum-powered quadcopters are available, but to overcome issues in the variations of engine speeds and latency, the time from sensor input to action, they often utilize variable pitch propellers with electronic actuators. These actuators allow for rapid, and subtle changes in propeller pitch adjusting for variable inputs from the sensors and the pilot. While gas-powered drones often provide greater thrust, for most applications modern drones are assembled using electronic motors. These electric motors are extremely efficient, respond rapidly to subtle changes in voltage by delivering predictable rotational speeds, all while being very lightweight. Coupled with highly efficient lithium batteries, these make for an ideal platform for building drones. The final component making these drones possible are advanced plastics and carbon fiber that now provide for very light-weight propellers that can take considerable abuse without fracturing or failing. When I grew up in the late 1960s and early 70s it didn’t take much to break that rubber band powered red plastic propeller that came with balsa wood planes (https://www.tintoyarcade.com/sky-streak-wind-upbalsa-wood-plane.html) of that era. Today I can crash my drone into the garage door at nearly full speed and all eight propeller blades remain scratch free. So next time you interact with a product and wonder why it doesn’t perform to your expectations, perhaps the technology has still not caught up to the intent of the product designers. Happy Holidays. { Add a Comment (https://technologyevangelist.co/2017/12/26/idea-technologyproduct/#respond) }

Ultra-low Latency Networking for Windows, Is There a Need? (https://technologyevangelist.co/2017/12/ 22/ultra-low-latency-networking-forwindows-is-there-a-need/) by scottcschweitzer on December 22, 2017

At a Flagg Management HPC on Wall Street (https://www.google.com/url? sa=t&rct=j&q=&esrc=s&source=web&cd=3&ved=0ahUKEwiOt6233p3YAhUQSt8 KHcXyBcgQFggzMAI&url=http%3A%2F%2Fwww.flaggmgmt.com%2Flinux%2Fi mages%2F11HPCLinuxattbro.pdf&usg=AOvVaw2ja9GRr-PuKv_1kjugET8G) event back in 2011, then CEO of Myricom Nan Boden, was pitching the concept of Ultra-low Latency for Windows. As a setup for this concept, she asked the audience how many used Linux for their low latency trading platforms. I was sitting in the back, but could easily see many, but not all hands were in the air. She then asked how many used Windows, and from the back, I saw a few lonely hands. Following the session, I commented on how few people were interested, and she said that was correct from my viewpoint in the rear. From her position on the stage nearly half had hands up, just in front of their bodies so few others could see. It was as if low latency trading on Windows were some dirty little secret. Later that year in August 2011 Myricom went on to release DBL for Windows (https://www.businesswire.com/news/home/20110801005648/en/MyricomIntroduces-DBL-Windows-–-Accelerated-UDP). While I remained there for another two years following this event, and handling sales for the Eastern region, all those hidden raised hands lead to very few sales, but why? Price was not the issue, DBL for Windows was extremely aggressively positioned against Linux. It wasn’t performance, while it was a measurable amount slower than Linux, it was still considerably faster than default Windows. We never were able to ferret out what the actual issue was. If you’re a Windows user craving ultra-low latency, please consider reaching out to me (mailto:[email protected]) and sharing your story. Thank you all for your time this year, and Happy Holidays. { Add a Comment (https://technologyevangelist.co/2017/12/22/ultra-low-latencynetworking-for-windows-is-there-a-need/#respond) }

Security: DARPA, HFT & Financial Markets (https://technologyevangelist.co/2017/12/ 18/security-darpa-hft-financial-markets/) by scottcschweitzer on December 18, 2017

Today nearly half of all Americans are invested in the financial markets. This past October the Dow Jones posted the “Pentagon Turns to High-Speed Traders to Fortify Markets Against Cyberattack (http://www.cetusnews.com/business/PentagonTurns-to-High-Speed-Traders-to-FortifyMarkets-AgainstCyberattack.H107qTxrCl6-.html).” The reporter had talked with a number of HighFrequency Trading (HFT) shops which had consulted directly with the Defense Advanced Research Projects Administration (DARPA). The objectives of these discussions were to determine how we could fortify the US financial markets against Cyber attacks. The reporter learned that the following possible scenarios were discussed as part of the “Financial Markets Vulnerability Project:” 1. Inject false information into stock data feeds 2. Flood the stock market with fake orders and trigger a market crash 3. Cripple a widely used payroll system 4. Credit Card Processors 5. Report fake news into systems used to algorithmically drive trading While protecting the US financial markets is something we expect of our government, the markets themselves are actually already insulated from outside attackers. The first two threats in the above list are essentially the same, placing fake orders into the exchange with no intent to honor them. To connect to an exchange’s servers a trader must be a member in good standing on that exchange and pay significant connection fees for their server to participate in that exchange. Traders place a very high value on their access to each exchange, and while HFT shops may only hold a security for a few millionths of a second, they understand the long-term value of losing access to an exchange. Most HFT shops have leased many 10GbE connections on multiple exchange servers, across multiple exchanges, and big bank’s dark pool, and very often Solarflare NIC cards are on both sides of these connections. So while it is technically possible for an HFT shop to inject enormous volumes of orders into one or more exchanges, a type of Denial of Service attack, using one or more physical ports on one or more exchange servers it could quickly result in financial suicide for that the trading firm. The exchanges and the Securities and Exchange Commission (SEC) don’t take kindly to trading partners seeking to game the system. Quickly the exchanges, and soon after the SEC, would step in and shut down inappropriate activity. *It should be noted that the above image was taken on December 6, 2017, in New York City’s Times Square. To further improve security for its trading customers later this month Solarflare will begin rolling out a beta of ServerLock™ which is a firmware update for these very same NICs powering the exchanges and HFT shops worldwide. With ServerLock™ the HFT shops and the exchanges themselves could rapidly pump the breaks on any given logical connection directly within the NIC hardware. This is the point at which DARPA and others should be interested. If the logic within the exchange were to detect and validate a threat they could then within a few millionths of a second install a filter into the NIC hardware to drop all subsequent packets from that threat. At that point, the threat would be eliminated, and it would no longer consume exchange CPU cycles. For HFT shops if they were to detect an algorithm had gone rogue they could employ ServerLock™ to physical cut a trading platform from the exchange without having to actually touch the platforms precious code. Much like throwing a cover over Schrodinger’s box, by applying the filter in the NIC hardware the trading platform itself remains intact for later investigation. Number three on the list above is crippling a widely used payroll processor like ADP who processes payroll checks for one out of six Americans? First ADP uses at least two different networks. One permits inbound payroll data from their client companies, over the public internet via SSL secured connections, and a second which is a private Automated Clearing House (ACH) network. The ACH network is a member network connecting banks to clearinghouses like the Federal Reserve. Much like the exchanges above, being a paid member of an ACH network then attacking that same network would not be a wise move for a business. As for the public Internet-facing connections that ADP maintains, they likely are practicing the latest defense in depth technologies coupled with least privilege in an effort to avoid the issues faced earlier this year by Equifax. Next, we have the Credit Card Processors also know as Payment Card Industry (PCI) players from Amex to Square who are fighting a never-ending battle to secure their systems against outsider threats. Much like the ACH network the PCI industry has its own collection of private networks for processing credit card transactions, ex. the Mastercard network, or Visa network, etc… These networks, like the ACH networks, are member networks, and attacking them would also be counterproductive. The world economy would likely not be in Jeopardy if at any point say the Amex or Discover networks were to stop processing credit cards for a few hours. We have seen the Internet websites of these providers, ex. Mastercard, have been targets of some of the most substantial Distributed DoS (DDoS) attacks the world has ever seen, and they’ve all faired it pretty well. Most have learned from these assaults how to further harden their networks. Who would have thought two years ago that “Fake News” could possibly have turned the tide of a US Presidential election, or be used as a tool to dramatically shift a financial market? While at DEFCON 2015 I watched as Charlie Miller and Chris Valasek presented their now infamous hack of a Jeep Grand Cherokee. At the start of their talk, Charlie joked that had they thought the wired article (https://www.wired.com/2015/07/hackers-remotely-kill-jeep-highway/) would have moved Chrysler stock more than a point or two he would have partnered up with a VC to fund shorting their stock. He said that had he done that he’d now be sitting on the beach of his private island now sipping his favorite frozen drink through a straw, rather than lecturing us. Charlie explained that he expected their announcement would be similar to Google or Microsoft announcing a bug, but he was very wrong. It led to a recall of 1.4 million vehicles and the stock dropped double-digit percentage points following the story and the recall. While this was real news, it was a controlled news release from someone outside the company. They could have easily made hundreds of millions of US dollars shorting the stock. Now what most people aren’t aware of is that there are electronic news systems that some HFT algorithmic platforms are subscribed to. Some of these systems even “read” tweets from key people (ex. our president) to determine if their comments might move a particular security or market in one direction or another. Knowing this, these systems can then be gamed by issuing false stories expecting that the HFT algorithms will then “read” these stories and stock prices will move appropriately. When retractions are issued later it might also be expected that they will place orders that would also benefit from these retractions. So how do we suppress the impact of “fake news” on our financial markets? These news services know that HFT systems trade on their output. Given that, they should be investing heavily in machine learning based systems to rapidly fact-check and score the potential truthfulness of a given story. For those stories that score

RSS feed RSS - Posts (https://technologyevangelist.c o/feed/)

Social Media (https://twitter.com/40GbE/) (https://www.linkedin.com/in/ScottSchweitzer/)

beyond belief, they should then be kicked to humans for validation or potentially be delayed until they are backed up by additional sources or even held until after the US markets close to further limit their impact. { Add a Comment (https://technologyevangelist.co/2017/12/18/security-darpa-hftfinancial-markets/#respond) }

Kernel Bypass = Security Bypass (https://technologyevangelist.co/2017/12/ 05/kernel-bypass-security-bypass/) by scottcschweitzer on December 5, 2017

As we move our performance focused applications to kernel bypass techniques like DPDK and Solarflare’s Onload this does not come without a price, and one component of that price is often security. When one bypasses the Linux kernel, they are also bypassing its security mechanisms (ex. XDP and NFTables, formerly IPTables). These security mechanisms have evolved over the past decade to ensure that your server doesn’t get compromised. Are they perfect no, software rarely is, but they are an excellent starting point to secure your Linux server. So as we move to kernel bypass platforms what options are available to us? We need to define lower level network security checkpoints that can be used as gatekeepers to keep the good stuff in and the bad stuff out. With one exception these are often hardware products that are managed using several different networking segmentation metaphors: micro, macro, and application which is also known as workload. Micro-segmentation is the marketing term that has been co-opted by VMWare to represent its NSX security (https://www.vmware.com/products/nsx/security.html) offering. When you’re a hypervisor company all the worlds a virtual machine (VM) so moving security into the hypervisor is a natural fit. VMWare then plays a clever trick and abstracts the physical network from the VM by installing a virtual network to which it then connects the VM. The hypervisor then works as the switch between the physical and virtual networks. To support coordinating workloads and security across multiple hypervisors running on different physical servers VMWare goes one step further and encapsulates traffic. This enables it to take traffic running on one virtual network and bridge it over the physical network to a virtual network on another host. So if your kernel bypass application can run from within a VM without having to rely on hypervisor bypass, then this model might work for you. Illumio has also attached itself to micro-segmentation, but rebranding it “smart micro-segmentation.” Our understanding is that they essentially run an agent that then programs NFTables in real time, so for kernel bypass applications this would offer no security. Macro-segmentation, as you might guess, means creating segmented networks that span multiple external physical network devices. This is the term that Arista Networks has chosen (originally they used micro-segmentation, perhaps until VMWare stepped in). Macro-segmentation is the foundation for Arista’s CloudVision (https://www.arista.com/en/products/eos/eos-cloudvision) line of products. While this too does an awesome job of securing your network it doesn’t come without cost, which is complexity. CloudVision connects into VMWare NSX, Openstack and other OVS DB based controllers to enable you to seamlessly configure various vendors hardware through a single interface. Furthermore, it comes with configuration modules called configlets for a wide variety of hardware that enables you to quickly and easily duplicate data center functions across one or more data centers. It also includes a configlet builder tool to quickly empower an administrator to craft a configlet for a device for which one does not exist. The last solution is application or workload segmentation. In techie terms, this is five-tuple filtering and enforcement of network traffic. Which to the layperson means opening the network packet up, inspecting the protocol it uses, along with the source and destination addresses and ports. Then taking these five values and comparing them to some collection of filter tables to determine the appropriate action to take on the packet. Today this can be done by Solarflare ServerLock (http://www.solarflare.com/hardware-security) NICs or applications like XDP or NFTables. ServerLock NICs do this comparison in 50 to 250 nanoseconds within the firmware of the NIC itself, entirely transparent to the server the NIC is installed in. In doing it this way the process of filtering consumes no host CPU cycles, is agnostic to the OS or applications running, and it scales with every NIC card added to the server. Packets are filtered at wire-rate, 10Gbps/port, and there can be one filter table for every locally hosted IP address with a total capacity exceeding over 5,000 filters/NIC. As mentioned, all of this filtering is done in the NIC hardware without any awareness of it by the DPDK or Onload applications running above it. So if you’re using DPDK or Onload, and the security of your application, or the data it shares, is of concern to you, then perhaps you should consider engaging with one of the vendors mentioned above. If you’d like to learn more about ServerLock, please drop me an email (mailto:[email protected]). { Add a Comment (https://technologyevangelist.co/2017/12/05/kernel-bypasssecurity-bypass/#respond) }

1 2 (https://technologyevangelist.co/page/2/) 3 (https://technologyevangelist.co/page/3/) 4 (https://technologyevangelist.co/page/4/) … 12 (https://technologyevangelist.co/page/12/) Older posts (https://technologyevangelist.co/page/2/)

© 2018 The Technology Evangelist - BlogKori Theme Developed by Tamal Anwar (http://www.tamalanwar.com)