Tuesday, March 27, 2012
Filter before export
I'm in the process of automating the export of a report to a PDF file.
The number of files is enormous and to prevent the query from executing over
and over again I was wondering whether the following scenario is possible.
- Get all records for a given report
- Filter by a specific value and export, repeat process for all necessary
values of that field.
Thanks for any input
Kind regardsYou could definitely do it programmatically from code that calls the RS Web
service...
--
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"WesleyB" <WesleyB@.discussions.microsoft.com> wrote in message
news:FB4303BE-E960-4493-B782-B1DFCA2F997A@.microsoft.com...
> Hi,
> I'm in the process of automating the export of a report to a PDF file.
> The number of files is enormous and to prevent the query from executing
over
> and over again I was wondering whether the following scenario is possible.
> - Get all records for a given report
> - Filter by a specific value and export, repeat process for all necessary
> values of that field.
> Thanks for any input
> Kind regards
>|||Hi,
Thanks for the answer.
Do you have any idea in which Namespace I can find the Filter property?
Thanks
"Wayne Snyder" schreef:
> You could definitely do it programmatically from code that calls the RS Web
> service...
> --
> Wayne Snyder, MCDBA, SQL Server MVP
> Mariner, Charlotte, NC
> www.mariner-usa.com
> (Please respond only to the newsgroups.)
> I support the Professional Association of SQL Server (PASS) and it's
> community of SQL Server professionals.
> www.sqlpass.org
> "WesleyB" <WesleyB@.discussions.microsoft.com> wrote in message
> news:FB4303BE-E960-4493-B782-B1DFCA2F997A@.microsoft.com...
> > Hi,
> >
> > I'm in the process of automating the export of a report to a PDF file.
> > The number of files is enormous and to prevent the query from executing
> over
> > and over again I was wondering whether the following scenario is possible.
> >
> > - Get all records for a given report
> > - Filter by a specific value and export, repeat process for all necessary
> > values of that field.
> >
> > Thanks for any input
> >
> > Kind regards
> >
> >
>
>|||I would use a parameter, passed in via URL. THen use the parameter to do the
grouping...
--
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"WesleyB" <WesleyB@.discussions.microsoft.com> wrote in message
news:FB4303BE-E960-4493-B782-B1DFCA2F997A@.microsoft.com...
> Hi,
> I'm in the process of automating the export of a report to a PDF file.
> The number of files is enormous and to prevent the query from executing
over
> and over again I was wondering whether the following scenario is possible.
> - Get all records for a given report
> - Filter by a specific value and export, repeat process for all necessary
> values of that field.
> Thanks for any input
> Kind regards
>|||Hi,
Wouldn't this trigger the query every time?
I'll try to give some more information.
We have about 1500 offices which all need a bunch of pre generated PDF
reports.
I want a report that returns only a couple rows per office to come back as 1
big Dataset that can be filtered clientside and then exported. So every
'filter' operation extracts the information for a specific office and then
exports it to a PDF. I do not want these queries to execute 1500 times
creating all the connection overhead.
eg.
Report returns 10 records per office
Returns 15000 records in the RS Dataset
The C# program (using Reporting Services WebService) filters out the 10
records and exports these to a PDF for every office
Kind regards,
Wesley
"Wayne Snyder" wrote:
> I would use a parameter, passed in via URL. THen use the parameter to do the
> grouping...
> --
> Wayne Snyder, MCDBA, SQL Server MVP
> Mariner, Charlotte, NC
> www.mariner-usa.com
> (Please respond only to the newsgroups.)
> I support the Professional Association of SQL Server (PASS) and it's
> community of SQL Server professionals.
> www.sqlpass.org
> "WesleyB" <WesleyB@.discussions.microsoft.com> wrote in message
> news:FB4303BE-E960-4493-B782-B1DFCA2F997A@.microsoft.com...
> > Hi,
> >
> > I'm in the process of automating the export of a report to a PDF file.
> > The number of files is enormous and to prevent the query from executing
> over
> > and over again I was wondering whether the following scenario is possible.
> >
> > - Get all records for a given report
> > - Filter by a specific value and export, repeat process for all necessary
> > values of that field.
> >
> > Thanks for any input
> >
> > Kind regards
> >
> >
>
>
Monday, March 26, 2012
FillFactor considerations
I have a table with about 200 million rows of data. I add a couple million rows of data each week to the table in a single load process. The table is used for reporting purposes only and there are never (not intentionally at least) any updates or deletes to the table. The data is always being added to the "end" of the table with the new AsOfDate being the main factor in the clustered index.
My question is this: Since I'm not "inserting" rows that would split pages, should I have my FILLFACTOR for the table set to 100, or am I missing something? I obviously want to save physical hard drive space, but I also don't want to slow down the import process.
BTW, I'm using SQL2000
If you are certain that your data loads is 'always' putting the new data at the end of the clustered index, you may have a good case for NOT leaving empty space on the data pages -and a FILLFACTOR of 100 would be in order.
FILLFACTOR 100 on Empty Table
I have a nightly process that starts with empty tables to create tables that
will be read-only durring the day. At the end of the process I would like
to have the indexes compacted to FILLFACTOR = 100. My process creates the
tables in one code module and finializes their processing in another code
module. I would prefer to keep the CREATE INDEX statements with the CREATE
TABLE statements (I know that my tables will populate faster without the
nonclustered indexes, but I don't care if it takes 5 minutes instead of 3;
reliable maintenance is a more important factor).
I'm thinking that if a table is empty that it shouldn't matter whether I
create the indexes with a FILLFACTOR = 1 or = 100; I'll just have one page
per index. And, once I start adding data, the FILLFACTOR is not a factor in
index maintenance. (BOL says FILLFACTOR is only for index creation, I'm
assuming that index pages will always be split 50/50 as records are
inserted.)
However, once my processing is done, and if I have used FILLFACTOR = 100,
then I can use DBCC DBREINDEX 'TableName' and all indexes will be compressed
to 100% FILL. This way I don't have to add every index in my early code
module to the final code module, thereby making maintenance more reliable.
So, to wrap it up, am I correct in my assumption that FILLFACTOR is not a
factor for insert performance when starting with a completely empty table?
--
Thank you,
Daniel Jameson
SQL Server DBA
Children's Oncology Group
www.childrensoncologygroup.org"Daniel Jameson" <djameson@.childrensoncologygroup.org> wrote in message
news:uwbh8yucGHA.536@.TK2MSFTNGP02.phx.gbl...
> So, to wrap it up, am I correct in my assumption that FILLFACTOR is not a
> factor for insert performance when starting with a completely empty table?
Yes; every observation in your post is correct.
Adam Machanic
Pro SQL Server 2005, available now
http://www.apress.com/book/bookDisplay.html?bID=457
--|||--
Thank you,
Daniel Jameson
SQL Server DBA
Children's Oncology Group
www.childrensoncologygroup.org
FILLFACTOR 100 on Empty Table
I have a nightly process that starts with empty tables to create tables that
will be read-only durring the day. At the end of the process I would like
to have the indexes compacted to FILLFACTOR = 100. My process creates the
tables in one code module and finializes their processing in another code
module. I would prefer to keep the CREATE INDEX statements with the CREATE
TABLE statements (I know that my tables will populate faster without the
nonclustered indexes, but I don't care if it takes 5 minutes instead of 3;
reliable maintenance is a more important factor).
I'm thinking that if a table is empty that it shouldn't matter whether I
create the indexes with a FILLFACTOR = 1 or = 100; I'll just have one page
per index. And, once I start adding data, the FILLFACTOR is not a factor in
index maintenance. (BOL says FILLFACTOR is only for index creation, I'm
assuming that index pages will always be split 50/50 as records are
inserted.)
However, once my processing is done, and if I have used FILLFACTOR = 100,
then I can use DBCC DBREINDEX 'TableName' and all indexes will be compressed
to 100% FILL. This way I don't have to add every index in my early code
module to the final code module, thereby making maintenance more reliable.
So, to wrap it up, am I correct in my assumption that FILLFACTOR is not a
factor for insert performance when starting with a completely empty table?
Thank you,
Daniel Jameson
SQL Server DBA
Children's Oncology Group
www.childrensoncologygroup.org"Daniel Jameson" <djameson@.childrensoncologygroup.org> wrote in message
news:uwbh8yucGHA.536@.TK2MSFTNGP02.phx.gbl...
> So, to wrap it up, am I correct in my assumption that FILLFACTOR is not a
> factor for insert performance when starting with a completely empty table?
Yes; every observation in your post is correct.
Adam Machanic
Pro SQL Server 2005, available now
http://www.apress.com/book/bookDisplay.html?bID=457
--|||Thank you,
Daniel Jameson
SQL Server DBA
Children's Oncology Group
www.childrensoncologygroup.org
Monday, March 12, 2012
Filegroup is full error
This is a server post even though I mention Analysis Services!
I am getting an error as follows when I try to process a cube:
Data source provider error: Could not allocate space for object '(SYSTEM
table id: -76163232)' in database 'TEMPDB' because the 'DEFAULT' filegroup is
full.;42000; Time:16/11/2004 7:05:34 PM
It looks like a SQL Server problem rather than AS.
The TEMPDB data file and transaction log are set on automatically grow and
unrestricted growth, and there is plenty of overall disk space (it's on a
RAID system).
Any ideas as to what the problem might be? Does the number 42000 mean
anything? A bad stripe has been suggested (i.e. hardware). I am going to
try adding another file to the PRIMARY filegroup but beyond that I am a bit
lost.
Hope someone can help.
LesSometimes it can not grow fast enough. Don't rely on autogrow if you know
you need more space. Manually grow the file(s) in that filegroup and try
again.
--
Andrew J. Kelly SQL MVP
"Les Russell" <LesRussell@.discussions.microsoft.com> wrote in message
news:8157C0BC-32E4-4C9C-85AF-50D8683A829B@.microsoft.com...
> Hello,
> This is a server post even though I mention Analysis Services!
> I am getting an error as follows when I try to process a cube:
> Data source provider error: Could not allocate space for object '(SYSTEM
> table id: -76163232)' in database 'TEMPDB' because the 'DEFAULT' filegroup
> is
> full.;42000; Time:16/11/2004 7:05:34 PM
> It looks like a SQL Server problem rather than AS.
> The TEMPDB data file and transaction log are set on automatically grow and
> unrestricted growth, and there is plenty of overall disk space (it's on a
> RAID system).
> Any ideas as to what the problem might be? Does the number 42000 mean
> anything? A bad stripe has been suggested (i.e. hardware). I am going to
> try adding another file to the PRIMARY filegroup but beyond that I am a
> bit
> lost.
> Hope someone can help.
> Les|||I had the same thing a while back if Im not mistaken. What it came out to be
was that tempdb couldnt grow fast enough to keep up with the demands. I
increased the size to 100 megs and havent had the problem since.
"Les Russell" <LesRussell@.discussions.microsoft.com> wrote in message
news:8157C0BC-32E4-4C9C-85AF-50D8683A829B@.microsoft.com...
> Hello,
> This is a server post even though I mention Analysis Services!
> I am getting an error as follows when I try to process a cube:
> Data source provider error: Could not allocate space for object '(SYSTEM
> table id: -76163232)' in database 'TEMPDB' because the 'DEFAULT' filegroup
is
> full.;42000; Time:16/11/2004 7:05:34 PM
> It looks like a SQL Server problem rather than AS.
> The TEMPDB data file and transaction log are set on automatically grow and
> unrestricted growth, and there is plenty of overall disk space (it's on a
> RAID system).
> Any ideas as to what the problem might be? Does the number 42000 mean
> anything? A bad stripe has been suggested (i.e. hardware). I am going to
> try adding another file to the PRIMARY filegroup but beyond that I am a
bit
> lost.
> Hope someone can help.
> Les|||Andrew,
Thanks to you and ChrisR. I think you are right.
I have redesigned the cube so that it is smaller, and it is OK now, but I
will check up how to grow the file manually for future reference.
Les
Filegroup is full error
This is a server post even though I mention Analysis Services!
I am getting an error as follows when I try to process a cube:
Data source provider error: Could not allocate space for object '(SYSTEM
table id: -76163232)' in database 'TEMPDB' because the 'DEFAULT' filegroup i
s
full.;42000; Time:16/11/2004 7:05:34 PM
It looks like a SQL Server problem rather than AS.
The TEMPDB data file and transaction log are set on automatically grow and
unrestricted growth, and there is plenty of overall disk space (it's on a
RAID system).
Any ideas as to what the problem might be? Does the number 42000 mean
anything? A bad stripe has been suggested (i.e. hardware). I am going to
try adding another file to the PRIMARY filegroup but beyond that I am a bit
lost.
Hope someone can help.
LesSometimes it can not grow fast enough. Don't rely on autogrow if you know
you need more space. Manually grow the file(s) in that filegroup and try
again.
Andrew J. Kelly SQL MVP
"Les Russell" <LesRussell@.discussions.microsoft.com> wrote in message
news:8157C0BC-32E4-4C9C-85AF-50D8683A829B@.microsoft.com...
> Hello,
> This is a server post even though I mention Analysis Services!
> I am getting an error as follows when I try to process a cube:
> Data source provider error: Could not allocate space for object '(SYSTEM
> table id: -76163232)' in database 'TEMPDB' because the 'DEFAULT' filegroup
> is
> full.;42000; Time:16/11/2004 7:05:34 PM
> It looks like a SQL Server problem rather than AS.
> The TEMPDB data file and transaction log are set on automatically grow and
> unrestricted growth, and there is plenty of overall disk space (it's on a
> RAID system).
> Any ideas as to what the problem might be? Does the number 42000 mean
> anything? A bad stripe has been suggested (i.e. hardware). I am going to
> try adding another file to the PRIMARY filegroup but beyond that I am a
> bit
> lost.
> Hope someone can help.
> Les|||I had the same thing a while back if Im not mistaken. What it came out to be
was that tempdb couldnt grow fast enough to keep up with the demands. I
increased the size to 100 megs and havent had the problem since.
"Les Russell" <LesRussell@.discussions.microsoft.com> wrote in message
news:8157C0BC-32E4-4C9C-85AF-50D8683A829B@.microsoft.com...
> Hello,
> This is a server post even though I mention Analysis Services!
> I am getting an error as follows when I try to process a cube:
> Data source provider error: Could not allocate space for object '(SYSTEM
> table id: -76163232)' in database 'TEMPDB' because the 'DEFAULT' filegroup
is
> full.;42000; Time:16/11/2004 7:05:34 PM
> It looks like a SQL Server problem rather than AS.
> The TEMPDB data file and transaction log are set on automatically grow and
> unrestricted growth, and there is plenty of overall disk space (it's on a
> RAID system).
> Any ideas as to what the problem might be? Does the number 42000 mean
> anything? A bad stripe has been suggested (i.e. hardware). I am going to
> try adding another file to the PRIMARY filegroup but beyond that I am a
bit
> lost.
> Hope someone can help.
> Les|||Andrew,
Thanks to you and ChrisR. I think you are right.
I have redesigned the cube so that it is smaller, and it is OK now, but I
will check up how to grow the file manually for future reference.
Les
Filegroup is full error
This is a server post even though I mention Analysis Services!
I am getting an error as follows when I try to process a cube:
Data source provider error: Could not allocate space for object '(SYSTEM
table id: -76163232)' in database 'TEMPDB' because the 'DEFAULT' filegroup is
full.;42000; Time:16/11/2004 7:05:34 PM
It looks like a SQL Server problem rather than AS.
The TEMPDB data file and transaction log are set on automatically grow and
unrestricted growth, and there is plenty of overall disk space (it's on a
RAID system).
Any ideas as to what the problem might be? Does the number 42000 mean
anything? A bad stripe has been suggested (i.e. hardware). I am going to
try adding another file to the PRIMARY filegroup but beyond that I am a bit
lost.
Hope someone can help.
Les
Sometimes it can not grow fast enough. Don't rely on autogrow if you know
you need more space. Manually grow the file(s) in that filegroup and try
again.
Andrew J. Kelly SQL MVP
"Les Russell" <LesRussell@.discussions.microsoft.com> wrote in message
news:8157C0BC-32E4-4C9C-85AF-50D8683A829B@.microsoft.com...
> Hello,
> This is a server post even though I mention Analysis Services!
> I am getting an error as follows when I try to process a cube:
> Data source provider error: Could not allocate space for object '(SYSTEM
> table id: -76163232)' in database 'TEMPDB' because the 'DEFAULT' filegroup
> is
> full.;42000; Time:16/11/2004 7:05:34 PM
> It looks like a SQL Server problem rather than AS.
> The TEMPDB data file and transaction log are set on automatically grow and
> unrestricted growth, and there is plenty of overall disk space (it's on a
> RAID system).
> Any ideas as to what the problem might be? Does the number 42000 mean
> anything? A bad stripe has been suggested (i.e. hardware). I am going to
> try adding another file to the PRIMARY filegroup but beyond that I am a
> bit
> lost.
> Hope someone can help.
> Les
|||I had the same thing a while back if Im not mistaken. What it came out to be
was that tempdb couldnt grow fast enough to keep up with the demands. I
increased the size to 100 megs and havent had the problem since.
"Les Russell" <LesRussell@.discussions.microsoft.com> wrote in message
news:8157C0BC-32E4-4C9C-85AF-50D8683A829B@.microsoft.com...
> Hello,
> This is a server post even though I mention Analysis Services!
> I am getting an error as follows when I try to process a cube:
> Data source provider error: Could not allocate space for object '(SYSTEM
> table id: -76163232)' in database 'TEMPDB' because the 'DEFAULT' filegroup
is
> full.;42000; Time:16/11/2004 7:05:34 PM
> It looks like a SQL Server problem rather than AS.
> The TEMPDB data file and transaction log are set on automatically grow and
> unrestricted growth, and there is plenty of overall disk space (it's on a
> RAID system).
> Any ideas as to what the problem might be? Does the number 42000 mean
> anything? A bad stripe has been suggested (i.e. hardware). I am going to
> try adding another file to the PRIMARY filegroup but beyond that I am a
bit
> lost.
> Hope someone can help.
> Les
|||Andrew,
Thanks to you and ChrisR. I think you are right.
I have redesigned the cube so that it is smaller, and it is OK now, but I
will check up how to grow the file manually for future reference.
Les
Friday, March 9, 2012
Filegroup Backup and Restore Issue
Peter,
All databases have a filegroup called the primary filegroup. The
existence of a filegroup therefore does not speed up backup and restore
per se. Using filegroup backups can be used to good effect to back up
unmanageable very large databases (VLDBs), by allowing you to split the
backup over several maintenance windows.
i.e. you could backup group1 on monday, group2 on tuesday, group1 on
wednesday, etc.
For smaller databases (say in the tens of Gigs), you should do full
backups. However, this can vary depending on your requirements.
Mark Allison, SQL Server MVP
http://www.markallison.co.uk
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602m.html
Peter wrote:
> Can filegroup speed up the process of backup and restore?
>
|||To add to Marks comments, filegroups allow you to make partial backups, and
restore partial backups. You should place each filegroup on a different
disk, then you can restore and backup the filegroups which map to a
particluar disk or raid array..
Wayne Snyder, MCDBA, SQL Server MVP
Mariner, Charlotte, NC
www.mariner-usa.com
(Please respond only to the newsgroups.)
I support the Professional Association of SQL Server (PASS) and it's
community of SQL Server professionals.
www.sqlpass.org
"Peter" <Peter@.discussions.microsoft.com> wrote in message
news:57086BDA-AA15-4F53-B40B-21D468959F0D@.microsoft.com...
> Can filegroup speed up the process of backup and restore?
>
filegroup auto-growth - know when it happens?
I'm running SQLServer 2000. I am in the process of looking over our current filegroups and providing reports based on file size increases.
One thing I'd like to know is when (date and time) a filegroup last "auto-grew". Is there a way to know when this occurs?
thanks,
Jason
SQL Profiler, Events, Database...
Data File Auto Grow
Data File Auto Shrink
Log File Auto Grow
Log File Auto Shrink
|||Derek,
Thanks for the info. I should have re-worded my question I think:
Is there a way to know when the date/time a filegroup last auto-resized? Let's say it auto-resized last friday, is this timestamp stored somewhere? Profiler would tell me only if I was running it at the time of the resize.
I'm trying to correlate the time of an error with the last time a large filegroup resized.
thanks
Jason
|||No. In general if you do not trace database activity it is lost. SQL Server has a builtin error log but it collects exceptions.|||Thanks for the info. Appreciate the help.
Jason
File transfer
thxCould you supply some more specifics?|||The specifies are:
I have to make a sp that will send/receive files to/from an ip address or ftp site.
The sp will query the information such as (filename, send_time, receive/send, servername, usernid, password, size_of_file, etc).
Once this info is queried, the script should transfer the file to respective location using, either exec xp_cmdshell or ne other way.
Since it's critical that the file being sent/received, must succesfully be sent/received. The script must also handle that.
This script will be running on SQL server. Any sample code?
thx|||I guess you're going to use ftp...
OK...build an ftp script...and execute it from a bat file with xp_cmdshell...don't forget to redirect the output...maybe load it to a log table...
I guess you'll interogate a directory..
Use xp_cmdshell 'DIR D;\whatever\*.*
and load that to a table to interogate that...
something like that?
Wednesday, March 7, 2012
File System Task Error - Process in use
In the integration services package i'm working on i connect to an excel database and have two data flows which deal with two work sheets in the excel spreadsheet. The data flows do some transformation and then write the data out to a SQL database (two differnet tables). This is all working great however what i want to do once both data flows are complete is to move the directory which contains the current file that is being looped through. At present there is a foreach loop going through a number of directories. I have tried to implement the File System Task object to move the directory but get the following error:
[File System Task] Error: An error occurred with the following error message: "The process cannot access the file because it is being used by another process.".
I've read a similar post where by the problem was due to not closing the excel connection before doing the File System Task. I cannot seem to find where i would be able to do this.
If anyone has any ideas i'm looking forward to hearing them.
Many thanks in advance,
Grant
i'm not sure what's causing your problem. however, my hunch is that a data flow task is still executing when control is passed to the file system task. you need make sure that both data flow tasks have completed before control is passed to the next task. one way to do this is to place both data flow tasks within a single sequence container, and then link this container to the next task. that way, control won't proceed to the next task until both data flow tasks have completed.
i hope this helps.
|||Hi Duane,
Thanks for the response. I had the same thought that the task hadn't released the file when it came to the moving of the directory. I have tried the suggestion of using the sequence container but i still get the same result. The move directory task still thinks it is being used by another process.
Any further suggestion would be welcomed. I thinking there must be a way to get round this or i'm sure it could be a common problem.
Many Thanks,
Grant
|||Hi Duane,Further to my last post. After some debugging i have found that it is the directory itself that seems to be in use by another process. I can remove the contents of the file no proble at all. Does this shed any more light on the matter?
Cheers,
Grant|||Hi,
I have found what i believe to be my problem and can't believe i didn't see it earlier. I'm doing all of my file and directory processing within a foreach loop. Hence when i come to moving the current directory i'm sure the foreach loop is still using the folder as it enumerated value. I need to be able to somehow call the move directory task outwith the loop so that it doesn't have control of the process. Could anyone suggest a way of doing this?
Thanks in advance.
Grant|||
if i understand you correctly, you want both data flows to finish before control proceeds to moving the files. also, you want this process to repeat once for each file that you have. i'm also assuming that you have two seperate directories of files that you want to process.
if my understanding above is correct, then you need to do the following: place two sequence containters inside a foreach loop container. connect an on success constraint from one sequence container to the other. there should be two seperate data flow tasks inside the first sequence container. only when this sequence container succeeds will control flow to the second sequence container. the second sequence container should contain two seperate file system tasks. only when this sequence container succeeds will control flow to the next iteration of the foreach loop container.
i hope this helps.
|||Hi,I'll clarify what i have set up at present.
There is a root directory in which a number of directories are stored. Within each of these directories are two files (one CSV and one XLS). We can ignore the CSV file just now. I have two data flow tasks, one which extracts transforms and loads data from one worksheet into a SQL table and another which does a similar process but from a different worksheet and to a different SQL table. Upon completion of both these tasks (which are now in a Sequence container) i want to move the directory containing both the files to an archive location. regardless of whether the file system tasks are in a sequence container there is still an error relating to the directory move. It still believes that the process is in use.
I still think that because the foreach loop is looping through the directories looking for each XLS file it is taking a hold on the directory itself and will not release it until the current directory until the next iteration of the loop.
With this thought in mind i thought that i could do something at the beginning of the loop to remove the directory that was referenced in the proevious iteration (possibly set a variable at the end of each iteration). The only problem i encouter when i try this is that because the variable has to be used as a source/destination value i cannot have this as a blank value when starting i get errors when this is attempted. I need to really skip the move directory on the first iteration and add an additional one at the end of the foreach loop.
My other thought is to just copy the directories during the loop and delete them after the loop has completed. I am aware however that using this method i run the risk that if something causes the package to crashh mid way through it might duplicate directories between the processing and archive directories.
Does this make any more sense.
Cheers,
Grant|||I have the problem resolved.
I have added the move directory task to the start of the foreach loop. A script bypasses the task on the first loop as there isn't a previous folder to archive. The archive folder path is assigned to the variable later in the loop. The next iteration the loop then has a folder to move in the variable which works no problem. After the loop is finished one final move task is added to deal with the final processed folder.
The reason it wouldn't allow an empty string as a connection source to begin with was because the "Delay Validation" property was set to false. If this is set to true the problem doesn't exist.
Thanks for all your help on this matter.
Grant
File system error: A FileStore error from WriteFile occurred
Error: Code: 0x00000000
Source: SRC Process dimensions Error:
Description: File system error: A FileStore error from WriteFile occurred. Physical file: \\?\<pathto filename> attribute.asstore. Logical file: . .
Errors in the OLAP storage engine: An error occurred while the <attribute id> attr
ibute' attribute of the <dimensionname> dimension from the '<database>' database was being processed.
Errors in the OLAP storage engine: The process operation ended because the number of errors encountered during processing reached the defined limit of allowable errors for the operation.
End Error
I encountered this while incrementally processing a dimension via AMO script in SSIS. Does anyone have ideas on what this error means and ways to avoid?
I unprocessed and reprocessed to resolve this.|||What do you mean by "Unprocess a cube"?
How did you unprocess a cube?
Thanks.
Chris
|||Description: File system error: A FileStore error from WriteFile occurred. Physical file: \\?\<pathto filename> attribute.asstore. Logical file: . .
Yikes. This has happened to me a few times within the cubes and dimensions. I also get the writefile error, which appears to be a bad reference to a file that no longer exists. I tried to use other processing methods ot process the dimension that was broken, and now I don't even get the errror anymore. Now the processing dialog box just hangs, saying that it is building a processing schedule. I do not want to unprocess the dimension, because I will have to reprocess the entire database. Does anyone have a clue as to how to avoid this, or why it happens? How could one fix it without unprocessing?
Thank,
Sally
|||That's a while ago. My problem was actually caused by the size of a dimension that I was processing.
It was just too big. Later I reduced the size, and the error disappeared.
Thanks anyway.
Chris
|||I have been experimenting with large dimensions (or at least what we consider to be large - ~100mm members), and have learned the string store (filename ends in .asstore) cannot exceed 4 gig. Is this the same thing you have run into, or have you found another limitation?
John
File system error: A FileStore error from WriteFile occurred
Error: Code: 0x00000000
Source: SRC Process dimensions Error:
Description: File system error: A FileStore error from WriteFile occurred. Physical file: \\?\<pathto filename> attribute.asstore. Logical file: . .
Errors in the OLAP storage engine: An error occurred while the <attribute id> attr
ibute' attribute of the <dimensionname> dimension from the '<database>' database was being processed.
Errors in the OLAP storage engine: The process operation ended because the number of errors encountered during processing reached the defined limit of allowable errors for the operation.
End Error
I encountered this while incrementally processing a dimension via AMO script in SSIS. Does anyone have ideas on what this error means and ways to avoid?
I unprocessed and reprocessed to resolve this.|||What do you mean by "Unprocess a cube"?
How did you unprocess a cube?
Thanks.
Chris
|||Description: File system error: A FileStore error from WriteFile occurred. Physical file: \\?\<pathto filename> attribute.asstore. Logical file: . .
Yikes. This has happened to me a few times within the cubes and dimensions. I also get the writefile error, which appears to be a bad reference to a file that no longer exists. I tried to use other processing methods ot process the dimension that was broken, and now I don't even get the errror anymore. Now the processing dialog box just hangs, saying that it is building a processing schedule. I do not want to unprocess the dimension, because I will have to reprocess the entire database. Does anyone have a clue as to how to avoid this, or why it happens? How could one fix it without unprocessing?
Thank,
Sally
|||That's a while ago. My problem was actually caused by the size of a dimension that I was processing.
It was just too big. Later I reduced the size, and the error disappeared.
Thanks anyway.
Chris
|||I have been experimenting with large dimensions (or at least what we consider to be large - ~100mm members), and have learned the string store (filename ends in .asstore) cannot exceed 4 gig. Is this the same thing you have run into, or have you found another limitation?
John
File system error: A FileStore error from WriteFile occurred
Error: Code: 0x00000000
Source: SRC Process dimensions Error:
Description: File system error: A FileStore error from WriteFile occurred. Physical file: \\?\<pathto filename> attribute.asstore. Logical file: . .
Errors in the OLAP storage engine: An error occurred while the <attribute id> attr
ibute' attribute of the <dimensionname> dimension from the '<database>' database was being processed.
Errors in the OLAP storage engine: The process operation ended because the number of errors encountered during processing reached the defined limit of allowable errors for the operation.
End Error
I encountered this while incrementally processing a dimension via AMO script in SSIS. Does anyone have ideas on what this error means and ways to avoid?
I unprocessed and reprocessed to resolve this.|||What do you mean by "Unprocess a cube"?
How did you unprocess a cube?
Thanks.
Chris
|||Description: File system error: A FileStore error from WriteFile occurred. Physical file: \\?\<pathto filename> attribute.asstore. Logical file: . .
Yikes. This has happened to me a few times within the cubes and dimensions. I also get the writefile error, which appears to be a bad reference to a file that no longer exists. I tried to use other processing methods ot process the dimension that was broken, and now I don't even get the errror anymore. Now the processing dialog box just hangs, saying that it is building a processing schedule. I do not want to unprocess the dimension, because I will have to reprocess the entire database. Does anyone have a clue as to how to avoid this, or why it happens? How could one fix it without unprocessing?
Thank,
Sally
|||That's a while ago. My problem was actually caused by the size of a dimension that I was processing.
It was just too big. Later I reduced the size, and the error disappeared.
Thanks anyway.
Chris
|||I have been experimenting with large dimensions (or at least what we consider to be large - ~100mm members), and have learned the string store (filename ends in .asstore) cannot exceed 4 gig. Is this the same thing you have run into, or have you found another limitation?
John
File system error: A FileStore error from WriteFile occurred
Error: Code: 0x00000000
Source: SRC Process dimensions Error:
Description: File system error: A FileStore error from WriteFile occurred. Physical file: \\?\<pathto filename> attribute.asstore. Logical file: . .
Errors in the OLAP storage engine: An error occurred while the <attribute id> attr
ibute' attribute of the <dimensionname> dimension from the '<database>' database was being processed.
Errors in the OLAP storage engine: The process operation ended because the number of errors encountered during processing reached the defined limit of allowable errors for the operation.
End Error
I encountered this while incrementally processing a dimension via AMO script in SSIS. Does anyone have ideas on what this error means and ways to avoid?
I unprocessed and reprocessed to resolve this.|||What do you mean by "Unprocess a cube"?
How did you unprocess a cube?
Thanks.
Chris
|||Description: File system error: A FileStore error from WriteFile occurred. Physical file: \\?\<pathto filename> attribute.asstore. Logical file: . .
Yikes. This has happened to me a few times within the cubes and dimensions. I also get the writefile error, which appears to be a bad reference to a file that no longer exists. I tried to use other processing methods ot process the dimension that was broken, and now I don't even get the errror anymore. Now the processing dialog box just hangs, saying that it is building a processing schedule. I do not want to unprocess the dimension, because I will have to reprocess the entire database. Does anyone have a clue as to how to avoid this, or why it happens? How could one fix it without unprocessing?
Thank,
Sally
|||That's a while ago. My problem was actually caused by the size of a dimension that I was processing.
It was just too big. Later I reduced the size, and the error disappeared.
Thanks anyway.
Chris
|||I have been experimenting with large dimensions (or at least what we consider to be large - ~100mm members), and have learned the string store (filename ends in .asstore) cannot exceed 4 gig. Is this the same thing you have run into, or have you found another limitation?
John
File system error: A FileStore error from WriteFile occurred
Error: Code: 0x00000000
Source: SRC Process dimensions Error:
Description: File system error: A FileStore error from WriteFile occurred. Physical file: \\?\<pathto filename> attribute.asstore. Logical file: . .
Errors in the OLAP storage engine: An error occurred while the <attribute id> attr
ibute' attribute of the <dimensionname> dimension from the '<database>' database was being processed.
Errors in the OLAP storage engine: The process operation ended because the number of errors encountered during processing reached the defined limit of allowable errors for the operation.
End Error
I encountered this while incrementally processing a dimension via AMO script in SSIS. Does anyone have ideas on what this error means and ways to avoid?
I unprocessed and reprocessed to resolve this.|||What do you mean by "Unprocess a cube"?
How did you unprocess a cube?
Thanks.
Chris
|||Description: File system error: A FileStore error from WriteFile occurred. Physical file: \\?\<pathto filename> attribute.asstore. Logical file: . .
Yikes. This has happened to me a few times within the cubes and dimensions. I also get the writefile error, which appears to be a bad reference to a file that no longer exists. I tried to use other processing methods ot process the dimension that was broken, and now I don't even get the errror anymore. Now the processing dialog box just hangs, saying that it is building a processing schedule. I do not want to unprocess the dimension, because I will have to reprocess the entire database. Does anyone have a clue as to how to avoid this, or why it happens? How could one fix it without unprocessing?
Thank,
Sally
|||That's a while ago. My problem was actually caused by the size of a dimension that I was processing.
It was just too big. Later I reduced the size, and the error disappeared.
Thanks anyway.
Chris
|||I have been experimenting with large dimensions (or at least what we consider to be large - ~100mm members), and have learned the string store (filename ends in .asstore) cannot exceed 4 gig. Is this the same thing you have run into, or have you found another limitation?
John