The .NET Platform
Development Tools
COM & COM+
Data Access
Web Development
XML Technologies
Windows Servers
Wireless & Mobile
Security issues
Design & Process
Career Development
Analysis & Comment
Disposable Objects
We found this book useful!
You are not logged in: login here to access all areas.
Inside the SQL Server team
Bren Newman from Microsoft's SQL Server development team talks to Matt Nicholson about what it's like to work at Microsoft, how Microsoft is tackling the security issues, and the technical advances in SQL Server that are to come.
Author: Matt Nicholson
Last updated: Aug 2002
Matt: What was it in version 7.0 that enabled you to
break into the enterprise market? Bren: The biggest reason was the size of the database.
The way we managed the structures of our database files in version 6.5
made it very difficult for us to maintain anything over about 10 or 20Gb.
We didn’t write for four processor machines, so our multi-processor
code was not optimal. Matt: Did SQL Server 2000 involve a rewrite of the engine? Bren: Moving from 6.5 to 2000 involved a complete rewrite.
In 6.5 everything was written to a 2K page, and we'd inherited a lot of
that code from Sybase. We actually started with SQL Server 1.1, using
the page structure of the original Sybase port. After that Windows NT
was coming along, so we decided we wanted to use a lot of the kernel abilities
that NT provided us with. It was around this point we decided to add our
own threading engine to the Sybase code, which was called UMS (User Mode
Scheduler). We told them we would take out their threading engine and
re-write that section ourselves. Matt: So you’re beginning to write specifically
for NT? Bren: Yes, we started chucking out stuff that wasn’t
specific. Although the team wasn’t very big, that was a big change. Matt: This was to allow you to cater for much larger
databases? Bren: Exactly, because a 2K page boundary limited us
in many ways. It stopped us doing certain things, especially with regards
to database management. So we made the move not just to 8K pages but to
row-level locking as well. Matt: It had become more critical, because you were
looking at a bigger amount of data. Bren: Exactly that. I’m glad you have that perspective
on it, because not everybody does. If you were using page locking it would
now lock an 8K page, which potentially means locking hundreds of rows.
So we decided to go for the row-level locking system. Some of the SQL
Server architects where from other large database vendors, bringing their
knowledge through to SQL Server with our goal being to get the enterprise
- in other words, large databases. Matt: Can you explain what exactly is the optimiser? Bren: The optimiser is a layer in SQL Server which works
out the best access plan in response to a query. It has to understand
what the best strategy is for a join. Is it best to just scan this table
and put it into a temporary area? To scan this table and join the results
together? Should I use mixed joins with a hashing algorithm? Should I
use a nested iteration loop, which is an old Sybase mechanism? The optimiser
looks at the data, looks at the statistics of past data operations, and
then works out what to do. It know hundreds of different ways to perform
an operation and uses a mechanism to decide which one it should actually
use. Matt: Does it learn the best procedures to use in specific
situations? Bren: It effectively learns from it’s statistics.
The database statistics are always changing, in that data is being input
and keys are changing. If your data starts becoming skewed it does understand
that, because the stats will change, but it isn’t a learning system
- it doesn’t remember all your queries and remember the best access
technique for them. Matt: What tools do you use in the SQL Server development
team? Bren: With version 7.0, everything was written in Visual
C++. Prior to that SQL Server was developed in pure, straight C using
Microsoft’s C2, and that was all DB library programming. C2 made
it quite difficult to grow though, and it became quite challenging when
more developers came on board. When we moved into the C++ environment
it made it a lot easier to check certain things, and inheritance became
easier - people could really start sharing code. Matt: And why not? Bren: Exactly. VB is really great, especially for the
UI. Matt: So it’s managed code written with Visual
Studio .NET? Bren: Yes. Sometimes it entails a rewrite, sometimes
it doesn’t. For example there’s the Replication Snapshot Agent,
which lends itself to being rewritten into C# because all the other components
use this one agent. It was the prime example of why to use C#, and it
proved very beneficial. Matt: What were the things that made it beneficial? Bren: Speed of development. We had an agent that took
years to write first time around, and then two developers rewrote it in
C# in about four months. And that involved taking what we had, looking
at it, understanding it, and understanding how people were going to use
it. Matt: How big is the development team? Bren: There’s probably about 200 developers in
the SQL Server team, as well as about 300 testers. Matt: How do you actually organise the team? Bren: Well firstly the team is broken down into product
units. So the relational engine, would have a product unit manager who
looks after this area, as well as liaising with all the other product
unit managers, and that’s a tough job. That manager probably has
some ‘leads’ under him, program managers who look after a
certain section. For instance, if I was working with replication, that
is divided into areas such as transactional replication and merge replication.
You would have a program manager for each of those areas who look at feature
sets and understand what competitors have and what the customer needs.
They work on those ideas, put them in writing, and create an overall plan
of action. The program manager works with a Dev manager. Their role is
to control a Dev team to implement those features and functions. Matt: So each unit has two managers? Bren: There’s actually three - the third is the
Test manager. He has to liaise with everyone to actually find out what
needs to be tested and how to write functional tests for certain features,
as well as performance and stress tests – real world tests. Matt: Do you use SourceSafe? Bren: We don’t use SourceSafe. Various teams within
Microsoft do use SourceSafe, but not us. Because our team comes from a
history of using its own tools, we’ve stuck to our own. There’s
no actual graphics involved in our tool: it’s just command-line
check-in, check-out. It’s served us well. Matt: Do you use the standard C++ compiler? Bren: The compiler is now the standard one, yes. Prior
to version 6.0 it was not. From version 6.0 we’ve been using totally
standard kit – standard compiler, standard linker, standard everything. Matt: On a different note, I’ve always wanted
to ask a Microsoft developer why so many security updates concern buffer
overflows. If you have an area of memory that you want to put something
into, it’s really simple to check the length of the thing you’re
putting into it. So why is it still happening? Bren: Well, the first issue concerns the language. C++
and C both lend themselves to these types of problems because they’re
pointer-based. Matt: But you can still check the lengths of things
you want to put into memory. Bren: Well, sort of. The problem is that most developers
at Microsoft don’t make their own calls. Let’s say I want
some memory. I don’t call memory allocation routines. Instead I
call a set of functions written for me in some header file. Someone’s
already written a very elaborate formula for doing all the checking and
other functions. Matt: Is that something specific to the SQL Server team? Bren: Actually, I think most teams have the same type
of design where they have all their core functions prewritten. So a lot
of developers become accustomed to just making a call to this trusted
mechanism. If that one trusted mechanism has some holes in it, everyone
gets those holes. Matt: But can’t that be fixed by fixing the mechanism? Bren: Yes, so what you will see now in the next Service
Pack of SQL Server is a massive amount of security fixing. Matt: This is a result of your security initiative? Bren: That’s right. Everyone had to go on training,
and I mean everyone. All Dev managers had to evaluate the code and evaluate
the people that are now working on that code, to be sure they better understand
secure programming techniques. Every single stored procedure – and
there’s a lot – had to be checked by multiple people. When
you check something in, your code has to be reviewed. So after you’ve
signed off your stored procedure, someone else does it again. Matt: Is it actually measured on an individual level,
so you can see who was responsible for a bug and who found it? Bren: We log every bug found in a bug database. Matt: Are people evaluated according to that database? Bren: Yes, so it’s a high-pressure environment.
It’s a database that gets monitored avidly by every single developer.
While you’re sitting there developing code, you’ve got that
bug database window open to spot any bugs that are assigned to you, and
if any turn up you want to get rid of them quickly. Matt: What are the implications of that change? Bren: Well, SP3 has been delayed. It’s a big,
important service pack from a security perspective. There are no new features:
there are standard bug fixes, but it’s really a deep-down security
release. Matt: So SP3 is a result of this security program? Bren: Yes, absolutely. Microsoft told us ‘drop
what you’re doing for the next version of Yukon, and only focus
on security.’ This was originally going to be for three months,
but it has turned out to be longer. This was not just an email from Bill
Gates - it was a massive ‘action item’. A laborious and tiring
one too. Matt: Going back to the development teams, how many
people would a Dev lead manage? Bren: Typically, Microsoft tries to keep small teams.
We find that managing more than four to five people becomes difficult,
so about four to five is average. Saying that there are smaller two-man
teams, and others that have ten people in them, but they are more unusual. A day in the life of a Microsoft developer Matt: What’s the working day like? Bren: It’s long! On top of having to come out
with a new build every day, we have these ‘It Does Work’ builds
that we call IDW builds. These builds are allocated a deadline, which
occur about every two weeks. Matt: What’s the average age of a developer there? Bren: SQL Server developers tend to be a little bit
older, in their late twenties. Matt: So do these people have families? Bren: Yes, a majority of them. Matt: Well, obviously they’re paid pretty well. Bren: Yes. I think Microsoft’s remuneration package
is quite well-known. They use the ‘two thirds’ rule, which
means they pay close to the top two-thirds of available salaries, so there
might be people who pay more. The incentives scheme is that every six
months you’re measured on your performance. You are given objectives,
which with a developer could be x number of bugs fixed, new features checked
in, communicating with your fellow partners, coming up with ideas, answering
developer forum emails, and so on. Communication is an important one because
without that, nothing happens. Matt: In terms of sharing knowledge, what kind of tools
do you use for managing the project as a whole? Bren: The program managers create skeleton-type specifications
that everyone has access to and comment on. These are stored as Microsoft
Word documents, although in the last two years these specs have migrated
on to the Internet and are sometimes stored as XML documents with diagrams
and so on. Matt: Do you use SharePoint Server? Bren: Yes we do. The teams themselves do a lot of internal
training, so there’s a lot of cross-pollination of skills. However,
between huge teams, like the replication team and the engine team, the
communication is at a higher level. It’s handled more between the
leads: they go and get the information they need to understand what the
engine can do, and then they speak to their developers. If it was just
individuals asking question all over place, it would get more stressful. Matt: What you’ve described is quite a hierarchical
structure. Bren: Although it’s sounds hierarchical and the
managers actually have the job title 'manager', the managers actually
check in code too. You can see testers talking to the overall product
manger, because there’s no formalisation to the structure. Matt: So you get around the politics? Bren: Yes. There’s no such thing as not being
allowed to talk to someone because they’re two levels up, or whatever.
Normally at Microsoft it’s all done by email as well – even
though the person’s only ten foot away – it’s just the
culture. Matt: We do that as well. I think the advantage is that
you can prioritise email. You don’t have to stop what you’re
doing to answer. It’s less disruptive. Bren: Yes, people who are developing don’t like
disruptions. The offices are all beautiful windowed offices – but
the people working there have blacked out the windows. The lights are
never on: there’s just a desk. You can talk to a lot of these guys,
but when they focus, they focus! Stored Procedures in managed code Bren: We are also enhancing programmatic access to SQL
Server. When we first brought it out, it consisted of DB libraries so
you had to make C function calls. Then we brought out our library for
Visual Basic. But right now it’s actually really easy: you design
your app and the database takes care of business. You can put all your
business logic into the table itself. You can set it up so a credit over
here will create a debit over there by using triggers. You can program
a stored procedure to take the data you are inserting and spread it across
all the different tables. The database can handle referential checks as
well. Matt: So you’re running managed code on the server? Bren: Yes. Matt: And does that mean that you would need to use
Windows .NET Server? Bren: Yes, because it requires the .NET Framework for
security. The biggest thing is the ‘sandbox’ technology in
.NET, in that all the security is built-in. Because if you’re going
to have these abilities at the engine level, it better be secure. Matt: So that’s going to open up a different way
of programming, because it means you can do sophisticated business logic
in the database itself? Bren: Right now, if you wanted the database to do sophisticated
business logic, beyond what you could do with Transact-SQL, you would
have to use a mechanism called Extended Stored Procedures. All that means
is that you have to write a DLL to the specifications that we give you,
and you can write it in whatever code you want, so long as you create
that DLL. We can then invoke that DLL from the database. The problems
with that are: one, it’s out of context so you have to switch over
to do that task; and two, if there are any graphics involved, it’s
going to die. Also, it relies on your own memory allocation techniques
and so on, which if they fail can quite easily cause problems such as
loss of data integrity. You could also cause SQL Server to stop altogether.
The next version of SQL Server Bren: David Vaskevitch has told us many times that we
need a common database store as well as common access methods - they all
need to be common. Because right now it’s not, you have registries
and you have Access and you have Active Directories and more - it’s
all over the show. Matt: You have the Exchange database and you have the
Windows file system. Bren: Yes, and in terms of a file system it should be
richer. That’s an area which Longhorn and versions after that are
definitely going to address: The ways in which the file system will be
richer. Matt: Will Yukon have some of these benefits? Bren: The objective of Yukon is not just to store relational
data in a data store - it’s more than that. We’ll get there;
but it’s not a guarantee in Yukon. Because we’re using the
CLR, the investment in testing is huge. They have to go out in the field
to collect all these databases, then create all these test beds around
them. And meanwhile the market is changing. 64-bit is becoming very important
- we already have a version of SQL Server 2000 running 64-bit and we’re
going to be releasing benchmarks in the next three weeks probably. The
beta of Yukon is expected by March 2003, so we’ll see around then.
The final release should be about a year away.
Bren:
I joined Microsoft as a consultant eight years ago, before joining the
Microsoft SQL Server development team as a test manager in replication.
My manager back then was Ron Soukup who's quite famous in the database
industry (he wrote Inside SQL Server). He started the database
industry within Microsoft and hired the likes of David Vaskevitch and
Paul Flessner, so an excellent guy to have as your mentor.
After shipping SQL Server 2000 I joined
the SQL Customer Advisory Team. There were about four of us worldwide,
which has since grown to a couple more, and we focus on key, large and
interesting SQL Server design wins.
Everything I’ve been working on for the last two years has been
very interesting. Prior to SQL Server 7.0 it was challenging to enter
the enterprise space, but once version 7.0 came along, we made strides
into that area.
When we started on version 7.0 the
first priority was to manage large sets of data. We had to rethink the
way we do maintenance and such like. This was the big one: we started
looking at databases of about 100, 150, 200Gb in size by the time 2000
came along.
Moving along, we decided that if we
were going to move to 6.5 (the next version) we wanted to make more use
of the functionality of NT, so there was a lot we added ourselves, such
as the Enterprise Management Tools. That was a big selling point.
We also changed the way you do locking
with version 6.5. We were still doing page locks using the original Sybase
code, but we decided to do a thing called ‘insert only row level
locking’. That’s not just a marketing term, we were able to
come up with a locking technology that doesn’t take a lock on a
single page but actually locks a single row. That was a big jump –
it doesn’t sound a lot, but it was!
After that there was a big time gap
between versions 6.5 and 7.0, purely because of what we had to do. We
decided that the 2K page limit wasn’t the way we wanted to go. We
wanted to go to a larger page structure, different indexes, different
types of join, change the optimiser – it was a big deal for us.
The biggest changes in SQL Server
2000 where in the optimiser. Given that there’s a lot of four processor
boxes out there, and even lots of eight processor boxes, we need to come
up with a very good strategy for optimisation. The growth of symmetrical
multi-processing boxes was a big jump. People think you can take the product
and run it on a multi-proc box and instantly get better performance, but
there’s a lot of programming involved to get it to work like that.
The ‘Create Process’, function call is not the be-all and
end-all, so we had to do a lot of tuning - particularly in the optimiser.
The reason we focused on the optimiser
in version 2000 was to understand how to work with large sets of data.
We tried to find the best strategy for joining large sets of data - not
just three to four tables, but ten table joins and more. We thought, now
that we’re at this level, let’s take on our competitors at
their own game: benchmarks. So we’re the leader in the field of
TPC (Transaction Processing Performance Council) benchmarks in certain
areas - even in very large-scale areas.
The big thing in version 2000 was
to scale-up and scale-out. We had a lot of competitors focus just on scale-up,
but we focused on both. The biggest step on scale-up was an eight-way
box, but we had a proposal from Unisys early on, who was going to build
a 32-processor box. So we braced ourselves and said, ‘okay, lets
have a look at our SMP (Symmetric MultiProcessing) design in terms of
our engine’, and it posed numerous challenges for us.
So our core rules of optimisation
changed. We had to physically understand things like NUMA (Non-Uniform
Memory Access - the way physical memory is shared), and understand that
it wasn’t quite NUMA. There were four processors in a block which
you can dock in and out. Each processor has an L3 level cache, so it doesn’t
always go through to main memory because that’s an expensive jump
in terms of performance. If you have eight of these processor blocks,
how do you get them to co-operate? And how do you scale up your optimiser?
So the question becomes the degree of parallelisation,
which can be challenging. When do you do things in parallel? How do you
know it’s not beneficial to actually go single thread? We spent
an inordinate amount of time just trying to understand that. We did a
great job and I think the two companies (Unisys and Microsoft) learned
a lot from each other.
We’re also running into people
who are putting 32 or 64Gb of memory on to the system. The optimiser then
has to understand how to handle that, so we had to make a couple of tweaks
there as well around what can be put into a cache and how to flush that
cache.
There are areas where we use assembly,
but it’s not even one percent of the total code. There are areas
within the optimiser and within the actual core engine that need to use
it. With SQL Server 2000 everything remains C++, but there are some Wizards
in the management tools that are actually written in Visual Basic: Conflict
Manager and things like that.
In the next version of SQL Server,
currently codenamed Yukon, there are a lot of items written in C#. These
items are not part of the core engine, so they’re not the optimiser,
and they’re not the relational engine, but a lot of the UI is written
in C#.
Bren:
It’s strange: there’s a lot of people who expect us to be
using tools that aren’t actually real. They think we must be using
magic. Nope, pretty much we use our own normal tools! Even beta versions
sometimes, although obviously we don’t use beta versions for release
products, or for service packs and things like that, because that’s
very controlled.
I think the biggest surprise people
get when they come and look at our development environment is the size
of the testing team. They cannot believe the ratio of two testers to each
developer. I know it’s expensive, and I know all the things that
go with that, but I think that’s the biggest surprise for people.
Most of the testers are also ex-developers, so a lot of the guys who are
getting on in years, like me, decide that maybe it’s time for the
younger guys to code.
So every time there’s a code
check-in, you can imagine what happens! We really scrutinise their code
to the extent a lot of them are hesitant to check it in – we go
through it line by line and if there’s a problem, we make it pretty
public.
Reporting to the Test manager, there
are test leads. Microsoft has a lot of leads. Similarly, we have Dev leads
reporting to the Dev manager. They focus on a certain section, so if they’re
working on transactional replication they will only focus on a single
area within that. This is a good strategy in terms of development, because
you’re focused on a finite set of jobs. The challenge is, that person
might not be aware of everything else that’s happening around them,
but you can get really good quality code out that way because that developer
works hand-in-hand with the Test lead.
Another thing is that Test and Dev
are at the same level: there’s no ‘I’m a developer and
you’re a tester’ feeling. At Microsoft they are paid the same
and treated exactly the same, so they’re the same quality people.
A lot of testers become developers, and that’s a strategy that’s
been used for five years now. A good tester will make a really good developer
because he has to learn all the mistakes. The process can be laborious
sometimes, but it has proven to be successful. All our top developers
are ex-testers or Product Support Services personnel – they’re
good because they feel the customer pain. It’s amazing because when
the program manager comes up with an over elaborate idea, they’re
quick to shoot the idea down. They say ‘that’s not supportable
- when a customer calls in, how’re they supposed to understand it?’
I think that strategy has been very successful.
Another question people often ask
is ‘how do you manage a product so big?’. And the answer is
the check-in process.
The process is pretty simple. You
check-out an area that you’re going to work on (which locks the
files); you make changes to the files; then you check the files in. When
you do that, the system automatically sends an email to an alias that
everyone monitors closely. At a particular time of night the source code
is then checked, and a version is pulled from the source and compiled.
It then goes through a standardised build process, an optimisation process
– the linker – and then after that it sends out a message
saying that the build is ready to be tested. Microsoft aims for a daily
build, which is quite a challenge.
Also, Microsoft is a competitive environment,
so the more bugs you find, the better it is for you if you’re a
tester. If you’re a developer, the less bugs you have the better.
So the process works pretty well.
It's even worse if you fix your bug
and your check-in causes a regression. That’s a developer’s
nightmare. So, when you check your file in, you get more than one person
to check it. And you have your own private build, including the full source
file on your own machine, so you can recompile with your own build first.
Then you run your own tests to check for basic errors, and only then do
you check it back in. You do all this because checking in a fix that causes
another bug is the ultimate sin.
If you follow the bug database throughout
a build, you notice a cycle where it starts off with a huge number of
bugs and slowly gets smaller. Then, as we come to release a beta version,
the bug count shoots up too - but then the fix rate goes up as well. A
lot of them are very small bugs, like spelling errors and things like
that, so developers fix most of them quite quickly. But a lot are substantial
and require a re-write or redesign.
This comes on to the security initiative
that you were talking about. There were a lot of areas that had been in
the code almost forever that we looked at again. The danger is that you
could be making a change to a core area of SQL Server that everyone uses,
so that better be stable.
There is then another build that we
give out to key customers, called an Early Adopter Build. That’s
another serious target, because we know that code is going to a customer.
The customer could be internal, another department within Microsoft, but
you constantly have this pressure.
When you eventually come close to
a beta build, that’s a three month push. During that time, although
people don’t get in until after 9am, many don’t leave until
the early hours of the morning - 2 or 3am. You receive dinner during the
day and they do advise you to go home. The Dev leads definitely try and
do that for their concentration levels, but because the environment is
so competitive, people want to get in their best code and their cool ideas.
As a developer I can come up with my own cool idea - there’s no
hierarchy, so you can go to anyone and talk about it, right up to the
highest level.
Saying that, there are people who
don’t work those hours. Some people work better if they come in
at 9am and leave at 5pm. In that time they can do what will take other
people until 10 at night to do - they’re just more efficient. Also,
a lot of the developers at Microsoft just want to be there. It’s
not that Seattle’s boring – they just love it. No matter how
much you advise them not to bring in sleeping bags, if you walk into the
offices you’re still going to see guys with them.
The other thing about Microsoft is
language barriers. We hire from across the world. Probably the majority
of developers at Microsoft are not American, so the communication can
be challenging, with Russians, Indians, Chinese and so on. Just communicating
your idea can be challenging. But at a technical level, telling people
what they need to develop, everyone knows how to do that, so there’s
no problem.
This also affects SQL Server support.
We have to write tests that work in all the languages, and that is very
challenging. What the SQL Server team does, and I think a lot of the other
teams too, is test six languages locally, and then send the rest off for
testing elsewhere. There’s a big centre for language testing in
Ireland. It’s a challenging job. They even have to test all the
error messages.
We also support Unicode, which is
difficult to test but has brought us lots of friends. There are a lot
of countries who aren’t going to use your software unless it’s
localised and can store their native characters. Challenging to code,
and lots of places that you have to go through and physically change the
code and the data structures. But once we had done it for version 2000,
that was it - the rest will be easy now.
Everyone in the team thinks the next
version is going to be better and more reliable. The grounding was done
with 7.0. That was the biggest change for us, to throw out all the Sybase
code and actually change the data structures, change the optimisers, change
the relational engine – everything was pulled out. At the moment
we can handle multi-terabyte databases, and in the future we’re
going to focus on much larger databases. You’re not going to be
constrained by disk size; it’s going to be much easier to store
large volumes of data.
I used to go to visit a customer and
they would say, “I have a one terabyte database”, and that
would make us go “wow!” But it’s becoming common now.
Just in the UK, a non-Microsoft person sent me a PowerPoint of ten customers
who I didn’t even know about, people using SQL Server with databases
of a terabyte and above and I’d never even heard of them!
It’s not just the data volume,
it’s the management of that data volume. It’s great to backup,
but can it be restored? The technology around restoring databases takes
a lot of time, it’s prone to failure.
As we moved from SQL Server 1.1, we’ve
added in all these things and made it easier to program against. With
the next version of SQL, developers are going to love us because we’ve
taken it to the ultimate level: you don’t have to write your stored
procedures or your data manipulation language in SQL any more. Personally
I’ll probably still use SQL because there are advantages to doing
that. However, if you want to write it in a managed code, in whatever
language you like, you can do that.
So it’s a very difficult way to do
it, but it’s there and people are implementing such things. They’re
using stored procedures that create actual processes, and there are some
people writing some very sophisticated apps that do this so I don’t
want to disrespect them, but the possibility is there for something to
go wrong. In the next version it’s basically sandboxed, so if it
does go wrong it’s ok: we’ll clean it up afterwards, we’ll
do the memory management, we’ll pull the thread. We’ll guarantee
those things through the CLR.
Matt: Quite a few people from Microsoft have talked about
future versions of Exchange using the SQL Server database engine technology.
Going beyond that, future versions of Windows are also expected to use
a common data store which will come out of the SQL Server engine. Can
you tell us about this?
With SQL the concern is how do we
store data and how do we make it common throughout - not from the file
system point of view, although I’m sure you’re right that
it will come eventually, but let's take it one step at a time. The biggest
things to update would be Exchange, Active Directory, and the registry
- those types of things. So is there a storage mechanism that can be common?
People always get this wrong. They say it’s going to be SQL Server,
but it’s not: it’s going to be an SQL Server-type engine.
Take SQL Server for CE for instance, it’s obviously not the entire
SQL Server, but it can understand DML (Data Manipulation Language), it
can understand any queries that go to it, it has components derived from
SQL Server and it runs in about 4Mb of space.
Now this engine and engines of its
type can be enhanced to handle such things as the registry and Active
Directory. The SQL Server data store, which is mostly a relational store
with lots of add-ons, lends itself to larger things than Exchange. However
the way in which Exchange has evolved to store data over the years has
led them to develop a database that’s rich for saving objects and
performing weird queries. Being relational, moving SQL Server’s
engine to Exchange would be a challenge. We have some guys within the
SQL Server department who have been brought in from the Exchange team,
and these guys are working on making our database richer so it can handle
those types of things. The database in Exchange is a derivative of Jet,
so we’re bringing that in and learning from it. We’ve learnt
a lot about making SQL Server rich enough for Exchange to use, and once
we can get Exchange running well on our database, it opens up a whole
new world for us.
We have a mechanism within SQL Server
that does an extremely good job of consistency checking, such that the
number of calls we get PSS on data corruption is extremely low - even
with multi-terabyte databases. So we know that we’re very good at
managing massive amounts of data. Hence Exchange again is a good target
for us. When a database goes down, the mechanism we use to store data
allows us to fix things and prevent them happening again easily. Exchange
benefits from the ability to index things on the fly, and we’re
going to bring that out in SQL Server. When we do that is anyone’s
guess, but it is an aim of ours.
Click here for our Privacy Statement. Copyright © Matt Publishing. All rights reserved. No part of this site may be reproduced without the prior consent of the copyright holder.
Development tools in use at Microsoft A day in the life of a Microsoft developer