This series of posts is essentially my presentation from the recent Data Weekender popup virtual conference – in blog form with some bonus additional content. This post focuses on the things you should consider before […]
At work we are seeing a burgeoning demand for Kubernetes test and development environment, as such we have been looking at simple and rapid ways to provision clusters. I have already blogged about the use […]
I have not blogged for a while, it was my hope to produce part 5 in the series of creating a Kubernetes cluster for production grade Big Data Clusters. However, there is a very good […]
The seed of the idea behind this blog post first germinated when I noticed the following yaml: Note the line that includes mergeTestResults, this got me thinking along the lines of running multiple tests in […]
I was originally going to cover storage in its entirety in a single blog post. However, as storage and Kubernetes is the cause of a tremendous amount of confusion in the Microsoft data platform community, […]
The previous post in this series covered Kubernetes cluster creation via Kubespray. It was my intention to cover off load balancing in this post, however at the time of writing when you create a SQL […]
Part 1 of this series covered the creation of the virtualized infrastructure for creating a Kubernetes cluster on. There are a variety of tools for building clusters, including Kops, Kubespray and Kubeadm. Kubeadm is perhaps […]
This blog post is the first in a series detailing how to build a Kubernetes cluster to deploy a SQL Server 2019 big data cluster to. For the purposes of learning and on-boarding there are […]
From a ‘Vanilla’ Kubernetes perspective; where all nodes in the cluster run on Linux, only containers based on Linux images can run. As of version 1.9 of Kubernetes, we are currently on 1.12 at the […]
In the previous post the scene was set for why container orchestration is required, what Kubernetes is and why the world of open source should be entered into with ones eyes wide open. This post […]
With the announcement of SQL Server 2019 big data clusters at Ignite, Kubernetes (often abbreviated to K8s) now stands front and center as part of Microsoft’s data platform vision. The obvious inference being that this […]
Where Were We ? In part I of this series I set the scene for why you would want to use docker and Jenkins for SQL Server continuous integration pipelines. The first post also covered […]
There seems to a great deal of interest in containers and Kubernetes at present, fueled by Microsoft hinting that Kubernetes has a big part to play in the future of the Microsoft data platform: Of […]
In the first post in this series I covered why you might want to use Jenkins as a CI engine and how to deploy to SQL Server running in a container using the ‘Sidecar’ pattern. […]
The mainstay of my presentation material this year has been my deck on continuous integration, Docker and Jenkins. For people who have not had the chance to see this presentation or have seen it and […]
In the previous part of this blog post I discussed how containers could be used to scale out a singleton work load. Where as my attempts to get my experiments to work ran into difficulties […]
I will forewarn readers of this blog post that this is ‘Conceptual’ in nature, due to the fact in my tests I was spinning up containers which then fell over with core dumps. Nonetheless, I […]
This post covers building a simple continuous integration environment using Jenkins and SQL Server data tools which is fully containerised. There are two github repositories associated with this post, the first contains the files for […]
Consider a scenario in which you wish to use DACPACs, but you want to spin up SQL Server in a container on Linux (say Ubuntu) because you wish to forgo the cost of having to […]
In this post I am going to demonstrate how to use one of Jenkins more powerful features ; its ability to create multi-branch build pipelines. Source Code Control and Branching 101 The very first step […]
In this post I will demonstrate how a neat trick brought to my attention by Niko Neugebauer can turn the processing of windowing functions from “Anti-scale” to scale-ability. First of all we need to create […]
In the world of continuous integration and delivery where we might want to perform numerous builds a day. Docker is ideally suited for spinning up environments and then tearing them down afterwards in use cases […]
I was fortunate enough to be selected to speak at SQL Saturday Dublin, the talk I gave was on leveraging the in-memory engine, the basic flow of the presentation is thus: I ask the audience […]
In this blog post I wanted to distill down the most fundamental points to consider when attempting to process a SQL Server workload in a scale-able manner. However, many of the principles I will outline […]
This post continues my work on the LMax disrupt or pattern, I’ve already covered this already, what I have not covered is: Spinlock profiling Wait statistic profiling How the in-memory engine now behaves with SQL […]
The aim of this blog post is twofold, it is to explain how: A “Self building pipeline” for the deployment of a SQL Server Data Tools project can be implemented using open source tools A build pipeline […]
Every so often a post appears on linked-in that serves as an oasis of information in a desert of spam. The article I speak of is “Non-Volatile Storage: Implications of The Data Center’s Shifting Center”. Its a god […]
A question I received following my pre-conference training day at SQL Bits and during my post-conference training day in Poland was how my material relates to SQL Server running in a virtualized environment. As there are figures […]
A lot of the work I have done over the last year has involved placing stress on the database engine via singleton inserts using a stored procedure that inserts rows in a loop under the […]
In my Saturday community day session at SQL Bits I mentioned an article by Linchi Shea from the sqlblog site in which he demonstrates that the overhead of 100% foreign memory access does not incur the […]
I will speaking at Join! Conference in Poland, on the Tuesday (May 10 th) I will be presenting a regular session on leveraging memory in SQL Server and on the Wednesday I will be delivering […]
This is probably one of the most unheralded new features of SQL Server 2016 which gets but a single bullet point in the CSS engineers “It Just Runs Faster” blog post series. Querying sysprocesses and […]
The Story So Far The graph below represents the throughput we managed to get ( from a warm buffer cache ) from the legacy database engine and the help of an in memory table as a scale-able […]
It is with great honor that I have the privilege of announcing that my pre-conference submission for SQL Bits XV has been accepted !. I will be putting on a days worth of training in […]
One of the many things I hope to get around to blogging about, time permitting, are the challenges of building “Web scale” platforms using SQL Server. Of the many challenges this presents, one is coming […]
Following some feedback from my last blog post: . . . what is inhibiting scalability ? As the degree of parallelism is increased, CPU saturation should be achievable unless some contended resource is being waited […]
There is a type of behavior in the database engine which undermines scalability, this is when multiple threads contend for a single resource, contention on the page free space bit map is the example that […]
This much delayed part 2 of my introduction to an SSIS ETL framework covers the latest iteration of my SSIS ETL framework which I will make freely available to anyone who wishes to use it. […]
The last blog post on spinlocks focused on the LOGCACHE_ACCESS spinlock and how pressure could be reduced on it by tweaking the CPU affinity mask. However, as is invariably the case with tuning you remove one bottleneck only for […]
I’m speaking at SQL Saturday Southampton on the 5th December, I look forward to seeing you there.
To date I have theorized about the behavior of certain types of spinlocks, now I have access to hardware which can put these under serious pressure I can quantify much of what I have blogged about. “A Tale […]
I’m writing this blog post in response to a question posed on twitter, asking which type of lock is preferable. The short answer is that at a high level it depends on your workload: OLTP […]
By “Core i Xeon” I am referring to the generation of Xeons ushered in by the “Core i” series micro-architecture, the first of which was Nehalem – Westmere being the same as Nehalem but with […]
From Anandtech: 3D XPoint has a lot to chew on. There hasn’t been an announcement this big in the memory industry since the invention of NAND in 1989 and while DRAM and NAND have improved and scaled a […]
This post is an addendum to my previous post, I hope that it answers some questions that might have popped up in readers heads whilst going through the last post: Q: Would affinitizing connections to A […]
One of my missions of late has been to obtain some time on a “Big box” in order to see how certain spinlocks behave under pressure, despite my best efforts I am not experiencing much […]
We are at the greatest inflection point in storage since the inception of the beast pictured below, the worlds first commercial hard drive in the form of the IBM RAMAC. I am of course speaking […]
This post will cover how to navigate the call stack in a trace captured using windows performance toolkit (WPT), firstly let me mention something prevalent across just about all event based profiling tools . . . Sampling As […]
It is with great honour that I can announce I have been selected to speak at SQL Saturday in Manchester on the 25th July, I look forward to seeing you there.
Join me at SQLSaturday Cambridge on September 12th. I look forward to seeing you there !
