by technology » Thu Aug 28, 2008 5:03 pm
Here are our responses to Chuck vdL's excellent comments.
(1) You're right: a test engine inside a browser is a client driving a remote
web server server, so it is indeed a "client/server" situation. The point
is that (a) eValid is an in-the-browser system, and (b) because of that it
represents an architectural departure from the other products mentioned.
(2) The user ("cfour3") complained that tests should be able to interact with
the GUI, and a main reason for doing this is to prevent tests from
becoming useless due to inconsequential changes. Most testers agree that
the capture/replay is less useful without the ability to have something
like "adaptive playback."
(3) Ajax applications in the eValid solution run normally in the browser, so
the complex internal state changes that Ajax introduces inside the browser
are "adapted to" automatically.
(4) On scaling we misspoke slightly: a Pentium IV @ 2.8 GHz with 2 GB running
XP gets 100+ in parallel; a Pentium Core2-Duo @ 2.54 GHz with 3 GB running
Vista gets 200+. As already pointed out, "your mileage will vary."
(5) The point about multi-threading is that eValid uses, in handling any given
web page, the same number of threads as does IE. Either adding more
parallel threads, or single-threading all the way, deviates from what the
browser does.
(6) The browser determines realism. Because eValid tests are done inside the
browser they are very close to a perfect emulation of a human user.
(7) Load test reporting can vary a great deal in the level of detail, and
there is a tradeoff: more detail means more overhead and less realism.
Full-session playback time or tiered times within a session seem to yield
good information, and have low enough overhead to preserve realism.
(8) You're right about SUT monitoring: eValid makes no attempt is to duplicate
the excellent IIS System Monitor capability. During server loading
experiments we study those reports very carefully as we vary the imposed
load on the SUT.
(9) Several different methods to feed data into individual parameterized
scripts can be used. Each user instance can have its own credentials, and
its own options. The ability to repeat any script N times in sequence is
also built in.
eValid Tech Support Team
