There seems to a great deal of interest in containers and Kubernetes at present, fueled by Microsoft hinting that Kubernetes has a big part to play in the future of the Microsoft data platform: Of […]
In the first post in this series I covered why you might want to use Jenkins as a CI engine and how to deploy to SQL Server running in a container using the ‘Sidecar’ pattern. […]
The mainstay of my presentation material this year has been my deck on continuous integration, Docker and Jenkins. For people who have not had the chance to see this presentation or have seen it and […]
In the previous part of this blog post I discussed how containers could be used to scale out a singleton work load. Where as my attempts to get my experiments to work ran into difficulties […]
I will forewarn readers of this blog post that this is ‘Conceptual’ in nature, due to the fact in my tests I was spinning up containers which then fell over with core dumps. Nonetheless, I […]
This post covers building a simple continuous integration environment using Jenkins and SQL Server data tools which is fully containerised. There are two github repositories associated with this post, the first contains the files for […]
Consider a scenario in which you wish to use DACPACs, but you want to spin up SQL Server in a container on Linux (say Ubuntu) because you wish to forgo the cost of having to […]
In this post I am going to demonstrate how to use one of Jenkins more powerful features ; its ability to create multi-branch build pipelines. Source Code Control and Branching 101 The very first step […]
In this post I will demonstrate how a neat trick brought to my attention by Niko Neugebauer can turn the processing of windowing functions from “Anti-scale” to scale-ability. First of all we need to create […]
In the world of continuous integration and delivery where we might want to perform numerous builds a day. Docker is ideally suited for spinning up environments and then tearing them down afterwards in use cases […]
I was fortunate enough to be selected to speak at SQL Saturday Dublin, the talk I gave was on leveraging the in-memory engine, the basic flow of the presentation is thus: I ask the audience […]
In this blog post I wanted to distill down the most fundamental points to consider when attempting to process a SQL Server workload in a scale-able manner. However, many of the principles I will outline […]
This post continues my work on the LMax disrupt or pattern, I’ve already covered this already, what I have not covered is: Spinlock profiling Wait statistic profiling How the in-memory engine now behaves with SQL […]
The aim of this blog post is twofold, it is to explain how: A “Self building pipeline” for the deployment of a SQL Server Data Tools project can be implemented using open source tools A build pipeline […]
Every so often a post appears on linked-in that serves as an oasis of information in a desert of spam. The article I speak of is “Non-Volatile Storage: Implications of The Data Center’s Shifting Center”. Its a god […]
A question I received following my pre-conference training day at SQL Bits and during my post-conference training day in Poland was how my material relates to SQL Server running in a virtualized environment. As there are figures […]
A lot of the work I have done over the last year has involved placing stress on the database engine via singleton inserts using a stored procedure that inserts rows in a loop under the […]
In my Saturday community day session at SQL Bits I mentioned an article by Linchi Shea from the sqlblog site in which he demonstrates that the overhead of 100% foreign memory access does not incur the […]
I will speaking at Join! Conference in Poland, on the Tuesday (May 10 th) I will be presenting a regular session on leveraging memory in SQL Server and on the Wednesday I will be delivering […]
This is probably one of the most unheralded new features of SQL Server 2016 which gets but a single bullet point in the CSS engineers “It Just Runs Faster” blog post series. Querying sysprocesses and […]
The Story So Far The graph below represents the throughput we managed to get ( from a warm buffer cache ) from the legacy database engine and the help of an in memory table as a scale-able […]
It is with great honor that I have the privilege of announcing that my pre-conference submission for SQL Bits XV has been accepted !. I will be putting on a days worth of training in […]
One of the many things I hope to get around to blogging about, time permitting, are the challenges of building “Web scale” platforms using SQL Server. Of the many challenges this presents, one is coming […]
Following some feedback from my last blog post: . . . what is inhibiting scalability ? As the degree of parallelism is increased, CPU saturation should be achievable unless some contended resource is being waited […]
There is a type of behavior in the database engine which undermines scalability, this is when multiple threads contend for a single resource, contention on the page free space bit map is the example that […]
This much delayed part 2 of my introduction to an SSIS ETL framework covers the latest iteration of my SSIS ETL framework which I will make freely available to anyone who wishes to use it. […]
The last blog post on spinlocks focused on the LOGCACHE_ACCESS spinlock and how pressure could be reduced on it by tweaking the CPU affinity mask. However, as is invariably the case with tuning you remove one bottleneck only for […]
I’m speaking at SQL Saturday Southampton on the 5th December, I look forward to seeing you there.
To date I have theorized about the behavior of certain types of spinlocks, now I have access to hardware which can put these under serious pressure I can quantify much of what I have blogged about. “A Tale […]
I’m writing this blog post in response to a question posed on twitter, asking which type of lock is preferable. The short answer is that at a high level it depends on your workload: OLTP […]
By “Core i Xeon” I am referring to the generation of Xeons ushered in by the “Core i” series micro-architecture, the first of which was Nehalem – Westmere being the same as Nehalem but with […]
From Anandtech: 3D XPoint has a lot to chew on. There hasn’t been an announcement this big in the memory industry since the invention of NAND in 1989 and while DRAM and NAND have improved and scaled a […]
This post is an addendum to my previous post, I hope that it answers some questions that might have popped up in readers heads whilst going through the last post: Q: Would affinitizing connections to A […]
One of my missions of late has been to obtain some time on a “Big box” in order to see how certain spinlocks behave under pressure, despite my best efforts I am not experiencing much […]
We are at the greatest inflection point in storage since the inception of the beast pictured below, the worlds first commercial hard drive in the form of the IBM RAMAC. I am of course speaking […]
This post will cover how to navigate the call stack in a trace captured using windows performance toolkit (WPT), firstly let me mention something prevalent across just about all event based profiling tools . . . Sampling As […]
It is with great honour that I can announce I have been selected to speak at SQL Saturday in Manchester on the 25th July, I look forward to seeing you there.
Join me at SQLSaturday Cambridge on September 12th. I look forward to seeing you there !
In this post I wanted to cover transaction log write performance in detail, some of the nuances of storage area networks that can work against you and how lowering write log latency can be more subtle than […]
Due to commitments to his startup venture Thomas Kejser has given me full access to his presentation material and asked me to present his Super Scaling SQL Server course going forwards. My intention would be […]
Most CPUs as we know them are based on the “Von Neumann architecture”: The Von Neumann architecture is based around the concept of the fetch execute cycle, instructions and data are fetched from memory, a […]
I’m delighted to announce that I have had my SQL Server Batch Engine and CPU Architectures deep dive session selected for SQL Saturday #413. Hopefully I will be able to add some new material to […]
At the end of the session I presented at the recent SQL Saturday in Portugal I had a chat with one of the attendees who mentioned that good xperf documentation is sparse and the question was posed: […]
In this post I would like to dig deeper into the NUMA awareness of the SQL Server batch engine. Firstly, can we infer anything from the SQL OS method calls with ‘NUMA’ in their name, […]
In a session I have presented both at SQL Konferenz in Germany and the recent SQL Saturday in Portugal, I cover CPU architectures and memory at level 400. As a ‘Hook’ with which to pique […]
There has been some recent discussion around whether the SQL Server batch engine is NUMA aware and does it support large pages, as I have been looking under the covers of this recently, I thought […]
According to general SQL Server ‘Lore’, hyper-threading works well for OLTP workloads and not so well for OLAP style workloads, but does this hold true for the batch engine ?, there is only one way […]
I am both honored and privileged to have been selected to present my “Deep Dive into SQL Server Batch Mode and CPU Architectures” at this event. I look forward to seeing you there if you are […]
This blog post will look at hyper-threading and the throughput it provides for OTLP workloads. How Does Hyper Threading Work ? This varies according to which generation of Intel processor you are talking about, for the purpose […]
This blog post consolidates most of the performance ‘Tweaks’ I have used in my blog posts to date. Many of the techniques I have used and blogged about relate to tuning SQL Server at scale, […]
To date I have almost exclusively focused on building and deploying DACPACs to conventional and container-ised SQL Server environments without any testing. This post extends the one on multi-branch pipelines and docker by adding two […]
Spinning up containers should generally be very rapid, however some container images, in particular those for SQL Server may take of the order of seconds to spin up. Take the following Jenkins pipeline groovy script […]
