[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

tcs

3/23/2007 2:57:00 AM

I am wondering if anyone can suggest a MAXDOP setting. I have a 64bit 8 CPU
server that will be under heavy load. It will have BCP running constantly
against it putting millions and millions of rows in it per day. The larger
tables will be partitioned by time. Users will also be querying against it
while the BCP is running.

Can anyone suggest a MAXDOP setting that will allow optimal performance a so
no one connection is starving others.

Thank You
12 Answers

Tony Rogerson

3/23/2007 8:08:00 AM

0

Hi tcs,

Parallelism comes from costly queries, so your first avenue is to go and
tune your queries so they don't do parallelism in the first place.

I've done a lot of work with BCP, well BULK INSERT of late and there are a
few things you need to watch out for when using it.

Are you using a BATCH_SIZE? If not, then the whole file load will be treated
as a single transaction which will affect concurrency and possibly causing
blocking.

If you use too smaller BATCH_SIZE and you get space waste, one of my tests
was to use a BATCH_SIZE of 1 because I was loading native data and that
causes just 1 row to be loaded per extent so for my 1 row (512 bytes say) a
whopping 64KBytes was used to store it - so, you can imagine how much unused
space I had left on my 1 million row load!

Have a look at my blog entry ->
http://sqlblogcasts.com/blogs/to.../archive/2007/03/11/bulk-insert-firstrow-and-batchsize-cause-very-poor-performance-and-significant-writes-to-ldf-an...

It might be best to BULK INSERT your data into a staging table in tempdb
first and then copy using (INSERT) in batches into your true application
table.

Tony Rogerson, SQL Server MVP
http://sqlblogcasts.com/blogs/to... (Ramblings from the field from a
SQL consultant)
http://sqlser... (UK SQL User Community)


"tcs" <tcs@discussions.microsoft.com> wrote in message
news:0975E8D6-2CFA-4C9E-AEFD-0014EA7CCF03@microsoft.com...
>I am wondering if anyone can suggest a MAXDOP setting. I have a 64bit 8
>CPU
> server that will be under heavy load. It will have BCP running constantly
> against it putting millions and millions of rows in it per day. The
> larger
> tables will be partitioned by time. Users will also be querying against
> it
> while the BCP is running.
>
> Can anyone suggest a MAXDOP setting that will allow optimal performance a
> so
> no one connection is starving others.
>
> Thank You

tcs

3/23/2007 1:10:00 PM

0

I thought having parallelism would help grabbing data across the partition.

My original question still stands though. What is a recommend MAXDOP
setting? Would it be 4 since I have 8 CPU's?

Thanks


"Tony Rogerson" wrote:

> Hi tcs,
>
> Parallelism comes from costly queries, so your first avenue is to go and
> tune your queries so they don't do parallelism in the first place.
>
> I've done a lot of work with BCP, well BULK INSERT of late and there are a
> few things you need to watch out for when using it.
>
> Are you using a BATCH_SIZE? If not, then the whole file load will be treated
> as a single transaction which will affect concurrency and possibly causing
> blocking.
>
> If you use too smaller BATCH_SIZE and you get space waste, one of my tests
> was to use a BATCH_SIZE of 1 because I was loading native data and that
> causes just 1 row to be loaded per extent so for my 1 row (512 bytes say) a
> whopping 64KBytes was used to store it - so, you can imagine how much unused
> space I had left on my 1 million row load!
>
> Have a look at my blog entry ->
> http://sqlblogcasts.com/blogs/to.../archive/2007/03/11/bulk-insert-firstrow-and-batchsize-cause-very-poor-performance-and-significant-writes-to-ldf-an...
>
> It might be best to BULK INSERT your data into a staging table in tempdb
> first and then copy using (INSERT) in batches into your true application
> table.
>
> Tony Rogerson, SQL Server MVP
> http://sqlblogcasts.com/blogs/to... (Ramblings from the field from a
> SQL consultant)
> http://sqlser... (UK SQL User Community)
>
>
> "tcs" <tcs@discussions.microsoft.com> wrote in message
> news:0975E8D6-2CFA-4C9E-AEFD-0014EA7CCF03@microsoft.com...
> >I am wondering if anyone can suggest a MAXDOP setting. I have a 64bit 8
> >CPU
> > server that will be under heavy load. It will have BCP running constantly
> > against it putting millions and millions of rows in it per day. The
> > larger
> > tables will be partitioned by time. Users will also be querying against
> > it
> > while the BCP is running.
> >
> > Can anyone suggest a MAXDOP setting that will allow optimal performance a
> > so
> > no one connection is starving others.
> >
> > Thank You
>
>

TheSQLGuru

3/23/2007 2:58:00 PM

0

I would test out between 3 and 6. That should prevent any one process from
starving others. But a blanket statement cannot be made given the
information given. It is up to you to test with your server, your data and
your queries. Sorry!

--
TheSQLGuru
President
Indicium Resources, Inc.

"tcs" <tcs@discussions.microsoft.com> wrote in message
news:D4764431-A607-42A5-9652-8EAC1306E5D7@microsoft.com...
>I thought having parallelism would help grabbing data across the partition.
>
> My original question still stands though. What is a recommend MAXDOP
> setting? Would it be 4 since I have 8 CPU's?
>
> Thanks
>


tcs

3/23/2007 3:34:00 PM

0

Yes I agree about the testing. I am just trying to see if anyone will say
that the default of 0 is NOT the best option. It is my understanding that
MAXDOP = 0 in this situation could starve other processes from "selecting".

thanks


"TheSQLGuru" wrote:

> I would test out between 3 and 6. That should prevent any one process from
> starving others. But a blanket statement cannot be made given the
> information given. It is up to you to test with your server, your data and
> your queries. Sorry!
>
> --
> TheSQLGuru
> President
> Indicium Resources, Inc.
>
> "tcs" <tcs@discussions.microsoft.com> wrote in message
> news:D4764431-A607-42A5-9652-8EAC1306E5D7@microsoft.com...
> >I thought having parallelism would help grabbing data across the partition.
> >
> > My original question still stands though. What is a recommend MAXDOP
> > setting? Would it be 4 since I have 8 CPU's?
> >
> > Thanks
> >
>
>
>

Tony Rogerson

3/23/2007 3:58:00 PM

0

SQL Server actually works out how many parallel streams to use based on
resource usage on the box so its dynamic anyway.

But, like I said - you need to address queries that are doing parallelism -
SQL Server is telling you that its a crap plan because of poor indexing or
stats or that you are processing a large amount of data in the query. What
do you mean by partitions?

Tony

--
Tony Rogerson, SQL Server MVP
http://sqlblogcasts.com/blogs/to... (Ramblings from the field from a
SQL consultant)
http://sqlser... (UK SQL User Community)

"tcs" <tcs@discussions.microsoft.com> wrote in message
news:C3EFFB82-DCD5-4AB6-A537-F12A1A343CF5@microsoft.com...
> Yes I agree about the testing. I am just trying to see if anyone will say
> that the default of 0 is NOT the best option. It is my understanding that
> MAXDOP = 0 in this situation could starve other processes from
> "selecting".
>
> thanks
>
>
> "TheSQLGuru" wrote:
>
>> I would test out between 3 and 6. That should prevent any one process
>> from
>> starving others. But a blanket statement cannot be made given the
>> information given. It is up to you to test with your server, your data
>> and
>> your queries. Sorry!
>>
>> --
>> TheSQLGuru
>> President
>> Indicium Resources, Inc.
>>
>> "tcs" <tcs@discussions.microsoft.com> wrote in message
>> news:D4764431-A607-42A5-9652-8EAC1306E5D7@microsoft.com...
>> >I thought having parallelism would help grabbing data across the
>> >partition.
>> >
>> > My original question still stands though. What is a recommend MAXDOP
>> > setting? Would it be 4 since I have 8 CPU's?
>> >
>> > Thanks
>> >
>>
>>
>>

KurtS

3/23/2007 6:35:00 PM

0

Although I will give you a straight answer, I think you should take
Tony's comments into careful consideration.

Set maxdop to 1 and forget about it. Query parallelism is of marginal
benefit in the best of cases. In many cases it has a negative effect
on performance. In a read-only data warehouse or reporting server
with very complex queries it can be of some benefit. In an OLTP
environment it is of very little use.

You are still going to get the the benefit of your multiple
processors. The difference is that each query will run on a single
processor instead of being broken into threads to be distributed
across multiple processors. The optimizer does not have to spend
cycles considering parallel plans and you won't see any more CXPACKET
waitstates.

If you do have a query or two that benefits from parallelism, set the
maxdop option in the query itself.

JXStern

3/23/2007 10:07:00 PM

0

On 23 Mar 2007 11:35:11 -0700, "KurtS" <ksurvance@sql-consulting.com>
wrote:

>Although I will give you a straight answer, I think you should take
>Tony's comments into careful consideration.
>
>Set maxdop to 1 and forget about it.

A refreshing opinion!

I mostly agree. A little testing I've done has never shown much more
than about 2x for higher settings, and I haven't investigated, but you
may use less memory, too, for a transaction running at maxdop=1,
getting more total throughput (other spids using other processors) for
your dollar.

Josh

Andrew J. Kelly

3/23/2007 11:48:00 PM

0

While I will agree that using parallel plans in a heavy OLTP env can have
relatively little benefit for queries and such you have to be careful about
simply setting MAXDOP to 1 at the server level, especially on Enterprise
Edition. That also restricts things such as Index rebuilds & creation,
DBCC's, Backup & Restores etc. And I can assure you that parallelism for
these operations can go a long way. I see you briefly hinted at something
like this but I see too many people set it to 1 and not realize all the
potential affects.

--
Andrew J. Kelly SQL MVP

"KurtS" <ksurvance@sql-consulting.com> wrote in message
news:1174674911.596368.129080@y80g2000hsf.googlegroups.com...
> Although I will give you a straight answer, I think you should take
> Tony's comments into careful consideration.
>
> Set maxdop to 1 and forget about it. Query parallelism is of marginal
> benefit in the best of cases. In many cases it has a negative effect
> on performance. In a read-only data warehouse or reporting server
> with very complex queries it can be of some benefit. In an OLTP
> environment it is of very little use.
>
> You are still going to get the the benefit of your multiple
> processors. The difference is that each query will run on a single
> processor instead of being broken into threads to be distributed
> across multiple processors. The optimizer does not have to spend
> cycles considering parallel plans and you won't see any more CXPACKET
> waitstates.
>
> If you do have a query or two that benefits from parallelism, set the
> maxdop option in the query itself.
>


JXStern

3/24/2007 8:13:00 PM

0

On Fri, 23 Mar 2007 19:47:53 -0400, "Andrew J. Kelly"
<sqlmvpnooospam@shadhawk.com> wrote:

>While I will agree that using parallel plans in a heavy OLTP env can have
>relatively little benefit for queries and such you have to be careful about
>simply setting MAXDOP to 1 at the server level, especially on Enterprise
>Edition. That also restricts things such as Index rebuilds & creation,
>DBCC's, Backup & Restores etc. And I can assure you that parallelism for
>these operations can go a long way. I see you briefly hinted at something
>like this but I see too many people set it to 1 and not realize all the
>potential affects.

Andrew,

I continue to be mystified when sp_who2 shows 20+ lines for a single
SPID on something like a four*singlecore processor box, or my
1*dualcore processor workstation. Are those tasks awaiting a
processor, or what?

If MAXDOP=1 is too strict, what would you say to MAXDOP=2? Since time
immemorial SQLServer has been *much* happier on at least a
two-processor box, but it was never entirely clear to me if that left
the OS more power, or if it was SQLServer internals that benefited
most directly.

Josh

Andrew J. Kelly

3/25/2007 5:02:00 PM

0

Josh,

The number of SPIDs you see indicate how many threads SQL Server spawned and
have no direct correlation to MAXDOP. MAXDOP specifies how many processors
in total any one piece of a plan can use at one time. The number of threads
may and often is different than the number of procs used. For instance you
may see ona 4 way box 13 threads which may break down to 3 per processor
with 1 as sort of a manager thread. They obviously can't all do work at
exactly the same time since there is only 4 procs. But that is how threads
work in general with most OS's. Each thread gets assigned a unit of work to
do and they will be swapped on and off the proc as appropriate. Changing the
MAXDOP may have an effect on how many are generated due to the availability
of the procs but that does not govern the thread count directly. Setting the
MAXDOP to 1 on a 4 proc box does not preclude SQL Server from using all 4
procs. It simply can't use them all at the same time for any one task. You
can still have 4 independent threads from 4 separate tasks all working at
the same time on all 4 procs. This allows for more concurrency. The correct
setting is what works best for your situation. What works for you may not
work for the server next to it as there are a lot of things that play a part
in how it is all used. Also keep in mind that SQL Server monitors the env at
the time each plan is run to see if it should even think about using more
than 1 proc or not. There are about 7 different factors that it takes into
consideration in deciding how many procs to use fora single task. One of
them is how busy the system is. So if you have a busy system to begin with
it may rarely use more than 1 proc regardless of the MAXDOP setting. YMMV
and the bottom line is you need to test it under real life conditions to
determine which is the best setting for your server and app.

--
Andrew J. Kelly SQL MVP

"JXStern" <JXSternChangeX2R@gte.net> wrote in message
news:oa1b03pbs3bfe3ppqtv294dthdbeih4mgk@4ax.com...
> On Fri, 23 Mar 2007 19:47:53 -0400, "Andrew J. Kelly"
> <sqlmvpnooospam@shadhawk.com> wrote:
>
>>While I will agree that using parallel plans in a heavy OLTP env can have
>>relatively little benefit for queries and such you have to be careful
>>about
>>simply setting MAXDOP to 1 at the server level, especially on Enterprise
>>Edition. That also restricts things such as Index rebuilds & creation,
>>DBCC's, Backup & Restores etc. And I can assure you that parallelism for
>>these operations can go a long way. I see you briefly hinted at something
>>like this but I see too many people set it to 1 and not realize all the
>>potential affects.
>
> Andrew,
>
> I continue to be mystified when sp_who2 shows 20+ lines for a single
> SPID on something like a four*singlecore processor box, or my
> 1*dualcore processor workstation. Are those tasks awaiting a
> processor, or what?
>
> If MAXDOP=1 is too strict, what would you say to MAXDOP=2? Since time
> immemorial SQLServer has been *much* happier on at least a
> two-processor box, but it was never entirely clear to me if that left
> the OS more power, or if it was SQLServer internals that benefited
> most directly.
>
> Josh
>