I'm in the process of reviewing the Domino Development training materials, and I think it needs an additional unit on performance. Here's my first stab at it -- I wonder whether I could get comments about any important items I left out, anything I got wrong, or anything I listed that isn't really critical. Your feedback is invaluable! Thanks.
INTRODUCTION
A common experience with new developers, is that they create an application that works great for them, but after it's in use for a while, they discover that three factors have caused their application's performance to degrade to the point of near uselessness. These three factors are:
Increasing number of documents.
Increasing number of users.
Huge numbers of deleted documents.
However, this slowdown isn't inevitable. If you're careful designing your application, you can retain reasonable performance with large numbers of documents and users, and avoid excessive deletions.
DESIGN FOR PERFORMANCE
The following isn't everything there is to know about building high-performing applications. We've concentrated on issues likely to be encountered by new developers, and those that are most likely to cause problems that aren't evident during development and testing, but only emerge later on.
Note: we use the term "expensive" here to denote something that will degrade performance -- something that takes a long time to execute or calculate.
The more fields you put on your forms, the more they will slow your application – not only is the form itself slower, but it takes longer to index views, so the application is slower to open and navigate.
“Computed for display” fields only affect the performance of the forms they are on – not view performance. Use Computed for Display when possible instead of Computed fields.
The more documents your database stores, the longer it will take to index your views. Databases containing millions of documents are possible, but only if the greatest care is taken in the design will an application perform acceptably with more than a hundred thousand or so.
The more you modify documents, the longer it will take to index views. Each time a view is used, the index must be brought up to date with any documents that were created or modified since it was last used.
The more you modify documents, the longer it will take to replicate your application. This is particularly a problem when you have many users with local replicas, using your application offline. If your application is tying up their replication, their email is delayed, they can't finish up and disconnect from the network, etcetera.
Therefore, avoid modifying documents unnecessarily.
Deleting a document and then recreating “the same” document is three times as bad in terms of performance, as just modifying the existing document.
NOTE: One frequent situation we see, are Notes applications that load their data from some other source – a text file or relational database. Every night, an agent runs. The agent deletes every document in the database, then reads the outside data source and creates a new Notes document for each record. This is the worst possible thing you can do for performance. There are algorithms available to do this synchronization with the least necessary change to your documents, which you can easily find by searching on the dW Notes forums (see Resources appendix).
More views will slow your application even if those views aren't used much. Think about the tasks your users need to perform and what views will be useful. Delete useless views.
The more columns there are in a view, the longer it takes to index the view. Include only those columns users need.
The more complex your column formulas are, the longer it takes to index the view. Consider moving particularly complex formulas into a computed field on a form, so that your views can just refer to the field -- particularly if the same column is used in multiple views. There's a tradeoff here because having more fields also slows things, so exercise judgment.
A view with a column re-sort is less expensive in terms of performance than two separate views, but a re-sort is not free. Avoid unneeded re-sorts, and use the column option “Defer index creation until first use” (new in version 8.0) to speed up views that have re-sorts.
A full-text index can take up a lot of room on disk, but it's usually worth it in terms of performance. That's because if you don't have the index, users are forced to use slower searching methods that tie up the server and take longer to give them results.
NOTE: A new database property in version 8.0 lets you turn off "full-text" searching of a database with no index. This decreases the amount by which users can tie up the server, but does not eliminate this problem as there's another searching technique that causes similar issues.
It's possible to create views that display the same document several times. For instance, you may have a multi-valued “Category” field and you categorize the view by this field, making each document appear under each category listed in that document. This is often a very useful thing to do, and we're not saying not to do it. But do be aware that this sort of view is expensive, in direct ratio to how many times each document appears in it.
Reader fields are somewhat expensive, especially in cases where the view contains many documents, but very few to which the user does have access. Of course, sometimes you must use Readers fields, but be aware of the cost, and consider ways to limit the number of documents or avoid using views (e.g. by mailing a doclink to the user who needs to see a document).
Categorized views are a better-performing choice when you have reader fields, provided you don't use the view option "Don't display empty categories." [note: confirm wording of this option].
The use of @Now or @Today in the selection formula of a view, or in the column formulas of views or folders, is very expensive, and should be avoided if at all possible. Investigate alternatives, such as:
Just sort the view in reverse order by date, so that today's documents are at the top.
Use a view categorized by date, and use @SetViewInfo to restrict the display to today's documents only.
The same kind of categorized view, embedded in a page, with a “single category” formula.
Use a folder instead of a view. Run an agent overnight which finds all the documents that should be in the view, and places them there. Set the accesses of the folder so that users may not manually add and remove documents.
Don't use the GetNthDocument method to iterate through a collection of documents. GetFirstDocument and GetNextDocument are better for this.
Don't automatically use the “NoCache” option on every @DbLookup and @DbColumn command. When you do this in a keyword formula, it slows your form significantly, and most keyword lists don't change every day. The cost of NoCache every time a document is opened, often isn't worth the convenience of not having to close and reopen the application to see an updated list once in a great while. Think about whether you really have to have that data "up to the second."
If your keyword formula calls @DbColumn, then uses @Unique to remove the duplicates, this is a sign that you're probably doing it wrong. Instead, use a categorized view or one with "Unique keys for ODBC Access" [confirm name of option] turned on, so that the keyword values are already unique. Once you get many documents, @DbColumn can take a long time, but returning 20 category headings takes about the same amount of time no matter how many documents are in the categories.
For many types of list fields, their keyword lookups don't have to be done when the document is opened in read mode. Use a keyword formula such as the following to avoid the performance hit when users are just viewing a document. [insert formula here]
TESTING
Finally, a key technique for making sure your design performs well going forward, is to actually test it under as realistic conditions as you can manage.
Estimate how many documents the database will ultimately have in it, and create twice that many test documents. It's usually possible to create realistic test data by creating a single test document, and a formula agent to make copies of the document assigning random values to certain key fields. [insert an example of such an agent here].
Note: When testing the design of an application that's already in use, make sure to do your testing on a copy of the application -- not a replica -- so that there's no chance of your test documents replicating into the production environment. It's also best to use a dedicated testing server, in case your application does anything that crashes the server or ties up the its resources.
Bear in mind that accessing data on a server is slower than accessing data locally, so performance testing done with a local database doesn't reflect performance in actual use.
Bear in mind that end users' machines and network connections may not be as fast as yours. Test performance on a lower-end system with a poor connection to see what the real user experience will be like.
It may be difficult to simulate conditions of high user load if you are doing all the testing by yourself. There are partner products for this, and [any other suggestions?]...
curtsey : IBM Site
INTRODUCTION
A common experience with new developers, is that they create an application that works great for them, but after it's in use for a while, they discover that three factors have caused their application's performance to degrade to the point of near uselessness. These three factors are:
Increasing number of documents.
Increasing number of users.
Huge numbers of deleted documents.
However, this slowdown isn't inevitable. If you're careful designing your application, you can retain reasonable performance with large numbers of documents and users, and avoid excessive deletions.
DESIGN FOR PERFORMANCE
The following isn't everything there is to know about building high-performing applications. We've concentrated on issues likely to be encountered by new developers, and those that are most likely to cause problems that aren't evident during development and testing, but only emerge later on.
Note: we use the term "expensive" here to denote something that will degrade performance -- something that takes a long time to execute or calculate.
The more fields you put on your forms, the more they will slow your application – not only is the form itself slower, but it takes longer to index views, so the application is slower to open and navigate.
“Computed for display” fields only affect the performance of the forms they are on – not view performance. Use Computed for Display when possible instead of Computed fields.
The more documents your database stores, the longer it will take to index your views. Databases containing millions of documents are possible, but only if the greatest care is taken in the design will an application perform acceptably with more than a hundred thousand or so.
The more you modify documents, the longer it will take to index views. Each time a view is used, the index must be brought up to date with any documents that were created or modified since it was last used.
The more you modify documents, the longer it will take to replicate your application. This is particularly a problem when you have many users with local replicas, using your application offline. If your application is tying up their replication, their email is delayed, they can't finish up and disconnect from the network, etcetera.
Therefore, avoid modifying documents unnecessarily.
Deleting a document and then recreating “the same” document is three times as bad in terms of performance, as just modifying the existing document.
NOTE: One frequent situation we see, are Notes applications that load their data from some other source – a text file or relational database. Every night, an agent runs. The agent deletes every document in the database, then reads the outside data source and creates a new Notes document for each record. This is the worst possible thing you can do for performance. There are algorithms available to do this synchronization with the least necessary change to your documents, which you can easily find by searching on the dW Notes forums (see Resources appendix).
More views will slow your application even if those views aren't used much. Think about the tasks your users need to perform and what views will be useful. Delete useless views.
The more columns there are in a view, the longer it takes to index the view. Include only those columns users need.
The more complex your column formulas are, the longer it takes to index the view. Consider moving particularly complex formulas into a computed field on a form, so that your views can just refer to the field -- particularly if the same column is used in multiple views. There's a tradeoff here because having more fields also slows things, so exercise judgment.
A view with a column re-sort is less expensive in terms of performance than two separate views, but a re-sort is not free. Avoid unneeded re-sorts, and use the column option “Defer index creation until first use” (new in version 8.0) to speed up views that have re-sorts.
A full-text index can take up a lot of room on disk, but it's usually worth it in terms of performance. That's because if you don't have the index, users are forced to use slower searching methods that tie up the server and take longer to give them results.
NOTE: A new database property in version 8.0 lets you turn off "full-text" searching of a database with no index. This decreases the amount by which users can tie up the server, but does not eliminate this problem as there's another searching technique that causes similar issues.
It's possible to create views that display the same document several times. For instance, you may have a multi-valued “Category” field and you categorize the view by this field, making each document appear under each category listed in that document. This is often a very useful thing to do, and we're not saying not to do it. But do be aware that this sort of view is expensive, in direct ratio to how many times each document appears in it.
Reader fields are somewhat expensive, especially in cases where the view contains many documents, but very few to which the user does have access. Of course, sometimes you must use Readers fields, but be aware of the cost, and consider ways to limit the number of documents or avoid using views (e.g. by mailing a doclink to the user who needs to see a document).
Categorized views are a better-performing choice when you have reader fields, provided you don't use the view option "Don't display empty categories." [note: confirm wording of this option].
The use of @Now or @Today in the selection formula of a view, or in the column formulas of views or folders, is very expensive, and should be avoided if at all possible. Investigate alternatives, such as:
Just sort the view in reverse order by date, so that today's documents are at the top.
Use a view categorized by date, and use @SetViewInfo to restrict the display to today's documents only.
The same kind of categorized view, embedded in a page, with a “single category” formula.
Use a folder instead of a view. Run an agent overnight which finds all the documents that should be in the view, and places them there. Set the accesses of the folder so that users may not manually add and remove documents.
Don't use the GetNthDocument method to iterate through a collection of documents. GetFirstDocument and GetNextDocument are better for this.
Don't automatically use the “NoCache” option on every @DbLookup and @DbColumn command. When you do this in a keyword formula, it slows your form significantly, and most keyword lists don't change every day. The cost of NoCache every time a document is opened, often isn't worth the convenience of not having to close and reopen the application to see an updated list once in a great while. Think about whether you really have to have that data "up to the second."
If your keyword formula calls @DbColumn, then uses @Unique to remove the duplicates, this is a sign that you're probably doing it wrong. Instead, use a categorized view or one with "Unique keys for ODBC Access" [confirm name of option] turned on, so that the keyword values are already unique. Once you get many documents, @DbColumn can take a long time, but returning 20 category headings takes about the same amount of time no matter how many documents are in the categories.
For many types of list fields, their keyword lookups don't have to be done when the document is opened in read mode. Use a keyword formula such as the following to avoid the performance hit when users are just viewing a document. [insert formula here]
TESTING
Finally, a key technique for making sure your design performs well going forward, is to actually test it under as realistic conditions as you can manage.
Estimate how many documents the database will ultimately have in it, and create twice that many test documents. It's usually possible to create realistic test data by creating a single test document, and a formula agent to make copies of the document assigning random values to certain key fields. [insert an example of such an agent here].
Note: When testing the design of an application that's already in use, make sure to do your testing on a copy of the application -- not a replica -- so that there's no chance of your test documents replicating into the production environment. It's also best to use a dedicated testing server, in case your application does anything that crashes the server or ties up the its resources.
Bear in mind that accessing data on a server is slower than accessing data locally, so performance testing done with a local database doesn't reflect performance in actual use.
Bear in mind that end users' machines and network connections may not be as fast as yours. Test performance on a lower-end system with a poor connection to see what the real user experience will be like.
It may be difficult to simulate conditions of high user load if you are doing all the testing by yourself. There are partner products for this, and [any other suggestions?]...
curtsey : IBM Site
Posts
