Very often, I'm using a
bootstrap file to set up my application when it's launched. I typically warm up the WakandaDB cache with entities, so I can do some benchmarks in kind of a real life situation later. The WakandaDB cache is filled, as queries are performed during the application life cycle. Basically, sequential queries (which means "queries performed on non-indexed attributes") load the entities into the cache, while indexed queries load only the index pages.
Here's what I'm doing to fill the cache with entities:
- Check the cache size and change it if needed. I set the size of the cache to 1.5 times the size of the data and the index files (.waData and .waIndx, in the "data" folder)
- Then I perform sequential queries on the classes I want to benchmark, so the cache is filled when I start the chronometer.
Here's the code of the bootstrap file doing this, with comments to explain the usage of some APIs. "initApp.js" is inside the "bootstraps" folder, which is not automatically created by Wakanda. "Bootstraps" is a required name, and all JavaScript files found in this folder will be executed at startup. You can also right click on file and select "Bootstrap" in the "Role" submenu (
see the documentation for more details).
// ============================== 1. Check/Change cache size
var gb = 1024 *1024 * 1024,
neededSize, currentSize,
dsName, msg;
// Get current value
currentSize = ds.getCacheSize();
// get the name of the application. For example "SuperWakCRM"
dsName = ds.getName();
// Get the size of the .waData file. Using a File object,
// built with the path to the appropriate file. Then we use
// the size property.
neededSize = File( ds.getDataFolder().path + dsName + ".waData" ).size;
// Add the size of the index file
neededSize += File( ds.getDataFolder().path + dsName + ".waIndx" ).size;
// Need more because of the way the cache organizes its data
neededSize *= 1.5;
// Change value only if needed. Notice that you can't _decrease_
// the size of the cache once the server is running, you can only
// increase it.
// Using Math.round and dividing by 100 to get only 2 decimal numbers.
console.log("Cache size: " + (Math.round((currentSize / gb) * 100) / 100) + " GB");
if(neededSize > currentSize) {
// With personal limits, actually ;-)
if(neededSize > (6 * gb)) {
msg = "Cache size required is too big (" + (needed / gb) + "GB). Ask your boss to change your computer.";
console.log(msg);
// Could also produce an error:
// throw msg;
} else {
db.setCacheSize(neededSize);
}
console.log("Increased cache size: " + (Math.round((ds.getCacheSize / gb) * 100) / 100) + " GB");
}
// ============================== 2. Fill cache with entities
// Performing sequential queries on non-indexed attributes
console.log('Loading the cache...');
ds.Employee.query('salary = 0');
ds.Company.query('postalCode = *');
ds.Country.query('isoCode = *');
// . . . other classes that will be used during the benchmarks
console.log('Loading the cache...done');
This code works perfectly, really. The problem is that with a 3GB data file (millions of entities in this application), the first queries are done in several seconds (around 25 on my computer). And while the bootstrap file is executed, you can't do a lot with the server. (Obviously the Studio and its graphical designers are OK.)
Actually, you can do very little.
Ok. Let's say you can do nothing.
Worse: You could even get the "Failed to reach server before timeout" alert.
However, in my situation, it's not mandatory at all for this code to be run before any other. I'm just filling the cache.
To speed up the startup of the application, I can use
Shared Worker. A Shared Worker is run in a separate thread, so the current thread - running the bootstrap file - is not blocked during the queries. And even easier: As I just want the worker to fill the cache, I don't need to communicate with it, to exchange messages, nor to do all those wonderful things you can do with Shared Workers. I just need to launch it.
So to speed up the initialization and to avoid funny alerts, here is what I am doing:
- Create a "loadCache.js' file that I put inside a "Workers" folder. Unlike the "bootstraps" folder, you can give whatever name you want to this folder. You can even put your JavaScript files anywhere you want in your project. You don't even have to be tidy. But I like when things are tidy. Freedom has a cost. Having tidy files is the limit.
- In this file, I cut and paste the code that warms up the cache.
- In initApp.js, I replace those queries with the launch of the Shared Worker:
// . . . previous code that sets the size of the cache . . .
// ============================== 2. Fill cache with entities
var ignore = new SharedWorker('Workers/loadCache.js', 'Load cache');
That's all.
Now, starting the server is way faster than it was before, and I won't this beloved "can't reach the server" alert anymore.
Which is sad, actually. I kind of liked it.
Comments
Ben Hayat
May 04, 2012
Permalink
The point...
So, if I understand your point correctly, you're having the system to do the "cache filling" procedure in a "Async" mode, rather than in "Sync" mode or perhaps in parallel mode, by using the SharedWorker thread? Was that the point? :-)
..Ben
David Robbins
May 05, 2012
Permalink
Love it.
Thibaud, thanks for this. :-)
Thibaud
May 05, 2012
Permalink
Actually, it is more about
Actually, it is more about "do in a SharedWorker what can take time to achieve while you don't need it _immediatly_ after the startup of your app ". I mean, it could be something else than warming the cache. And in most applications, you don't need to warm the cache, it is filled while users are requesting data. I'm doing this because I'm benchmarking some parts, and need the cache to be filled (don't want to wait the first time I launch my testing ;-))
Add new comment