As I mentioned in my last blog post , and promised to provide more details on, I have been running into problems getting one of the larger Domino.doc (DDM) File Cabinets to extract using the DDM Migrator from IIUI.
The first File Cabinet exported without any problems using the latest version of the Migrator, so I was a bit surprised at the problems I ran into with this latest File Cabinet.
The problem that was occurring was a “Notes error: invalid or nonexistent document” message while using the XML export option and migrating the File Room DDM Taxonomy.
There is a great quote from author and motivational speaker Jim Rohn, that goes;
“To solve any problem, here are three questions to ask yourself:
First, what could I do?
Second, what could I read?
And third, who could I ask?”
What Could I Do?
My first attempt to debug the issue was to enable the Debugging option in the Migrator and re-run the DDM Taxonomy export.
The export process ran, resulted in the same error as before, but this time created a large number of log file entries.
That lead me to the second question.
What Could I Read?
The debug log files right? Examining the debug log files showed a large number of entries, unfortunately the resulting log entries didn’t list any errors. They simply stopped mid run with the final entry listing “1. Create Binder Stub – Begin”. No subsequent entries. No errors logged. The log entries didn’t reference a document UNID, or any details on what Binder was being processed. So no ability to track down the specific Binder that was being processed or what the invalid document might be.
That lead me to the third question.
Who Could I Ask?
At this point I reached out to Matt Mangels at IIUI ( http://www.iiui.com ), the creators of the Migrator, to ask for his assistance troubleshooting the issue.
One suggestion was to try and narrow down the export by processing a single Binder at a time, but the File Cabinet I was dealing with contained 2100 Binders, doing them one at a time to find the affected Binder would take forever.
Matt took a look at the log file entries being created and suggested that the issue might be due to a couple of possibilities;
1. exporting a single binder which was the default binder
2. database corruption in the DDM File Cabinet databases
3. damaged folder linkages in the DDM File Cabinet databases
Matt related that it was extremely rare to have issues during the taxonomy export process. Errors normally cropped up in the documents themselves.
Matt recommended I look into the above suggestions and if that didn’t help he could try to add some extra logging to the file to get a handle on what binder was being processed when the error occurred.
What Next?
The first option wasn’t the issue, as I was exporting an entire File Cabinet with multiple binders not a single Binder.
For the second and third options Matt suggested that I run fixup, and compact on the File Cabinet databases and Rebuild Folders.
I contacted the client to confirm that it was ok to perform these steps on their DDM File Cabinet. After receiving their ok to proceed in off-hours I did the following;
1. made a backup replica of the DDM File Cabinet databases (just in case)
2. ran fixup and compact on the two databases making up the DDM File Cabinet
3. opened the hidden (Binders by Document Database) view in the File Cabinet and selected the Rebuild Folders button to rebuild the File Cabinet folders. (This Rebuild Folders process took hours to run.)
At this point I tried exporting the Taxonomy again and again received the “Notes error: invalid or nonexistent document”. No luck.
The next thing I tried was purging deletion stubs from the File Cabinet databases. I have seen instances in the past where an agent can get a handle on a deletion stub in a database as part of a collection and then when it tries to access the stub results in an uncaught error. After purging the deletion stubs I tried exporting the taxonomy again, and this time it worked without an error. I thought “aha problem solved”. But alas it was not to be.
Once I tried exporting the document stubs I received the same error “Notes error: invalid or nonexistent document”, and only 127 documents were exported vs. an expected 6,500.
Now it Gets Strange:
Now comes the really strange part. The log file in the migrator reported over 15,380 error references. Why is that strange you ask? Well first of all because the File Cabinet only contains 6,500 documents, and debugging was turned off, so only minimal logging was occurring. But the truly strange part was every one of the 15,380 error references pointed to the exact same damaged document in the export working directory.
So somehow, it seemed to me like one of two things were happening;
a). a single document was causing a loop to occur that was resulting in the large amount of errors and an eventual crash
or
b). there were multiple errors being generated for different documents, but the document pointer information from the last document processed was being cached and reproduced in subsequent log entries
At this point I reached out again to Matt to say that I was going to need that extra logging information that he had offered to add to the Migrator.
Given that this error was happening in a different spot than the previous one Matt got on a web meeting with me to review the issue and take a look at the environment.
We verified that the damaged export document that all of the errors were pointing to was a document in DDM.
We then focused in on the specifics of the document and in particular the binder category, binder name, and document title as Matt described that this was most commonly the source of problems when doing an extract.
From reviewing the file path that was being generated by DDM Migrator we noticed a discrepancy.
The file path that DDM Migrator was trying to write the file to was as follows;
D:\MWDDMFiles\Information Control\Manufacturing\Manufacturing\Assembly Instructions – Fire\AVXC Upgrade\TOP HOUSING\CTA-052-00-(SS)-AX\AI-0031, (I05) [cta-052-00- (SS)-AX AVXC TOP CTA ASSY, ALPE AL OR SS] .AI-0031+ (I05) [cta-052-00-_1.0.pdf
Comparing this to the source DDM data that was being used to generate the file path; DDM_TAX_LIBRARY: Information Control
DDM_TAX_ROOM: Manufacturing
DDM_TAX_CABINET: Manufacturing
DDM_TAX_CATEGORY: Assembly Instructions - Fire\AV/XC Upgrade\TOP HOUSING\CTA-052-00-(SS)-AX
DDM_TAX_BINDER: AI-0031, (I05) [cta-052-00- (SS)-AX AVXC TOP CTA ASSY, ALPE AL OR SS] .ai
DDM_PROFILE_TITLE: AI-0031, (I05) [cta-052-00- (SS)-AX AVXC TOP CTA ASSY, ALPE AL OR SS].pdf
We gleaned a few things;
1. the output file path wasn’t a match to the source DDM document data (Library \ Room \ Cabinet \ Category \ Binder \ Title)
2. the overall path was longer than 250 characters which means the final file name would be shortened (hence the cta-052-00-_1.0.pdf extension instead of the full title)
3. the file path being generated was missing the binder name as a folder leve.
The Solution:
Item # 3 was the key thing that looked out of place. For some reason the file path was missing the file folder that should correspond to the Binder Name level and was instead concatenating the file folder and the file name.
The offending culprit looked like it was the binder name ending in ” .AI”. The Migrator will truncate the path at 216 Matt figured that the presence of both the space and the period at the end of the file path being generated by Migrator was causing the problem. Coupled with the fact that Migrator will try to shorten a patch to keep it under a 250 character limit. A folder created in Windows will strip off a space and a space and period at the end so it is likely that Migrator was creating a folder ending in ” .” which was being stripped off by the file system, and then wasn’t able to find a matching folder when it tried to process the record.
Armed with this example Matt went back to reproduce the issue on his end and look to handling trapping a file path that ends in both a space and a period.
Later on that day Matt emailed me with a patched copy of the Migrator that resolved the issue.
My thanks to Matt for pushing out a fix so quickly as it allowed me to get moving again with the migration.
I installed the updated copy of the IIUI DDM Migrator on the server, started the export process. Drum Roll Please….
Success! 6,500 exported documents created successfully along with the DDM Taxonomy.
The only issue were 5 documents referenced in the log saying that they couldn’t be exported because they did not have a Title assigned in DDM.
No problem.
These can be manually transferred following the import and an appropriate title assigned.
Next Steps:
Import this File Cabinet extract into DOCOVA using the DOCOVA Migration Module.
