Testing with Rspec, Cucumber, Selenium, Watir & Buzz around BDD, TDD

I was lucky to be working with some folks this past week, who are working on a bleeding edge commerce framework and this particular blog is more on the testing frameworks for front end.

I will soon have an article on building a front end store using ROR for Websphere commerce and it will be really useful to automate the test scripts using RSpec and one of the automation tools out there.

This is meant to be 101 for these concepts and I have added links below for deep diving.

BDD and TDD are paradigms essentially bringing the prominence of writing test cases before development and also making it easier on non coders to understand the test cases.

TDD: Test driven development
1. Writing automated test scripts before actually starting to write codes.
2. It is a great idea using any programming language but traditionally development with languages such as Java was pretty high hence it was skipped as the larger development life cycle was blamed for it.
But It makes great sense for using this approach for ROR (Ruby).
3. Test driven development brings lot of clarity to the design in the very early stage of the development cycle.

e.g.

BDD: Behavior Driven Development
1. It extends the concept of the TDD and takes it one level further as it makes writing test scripts in a very expressive way and no programmatic way.
2. It really helps non programmers (BA, QA, PM) to understand the test cases as they are expressed in a simple English like language.
3. Using 'should' when describing the behavior of software to help clarify responsibility and questioning the requirements.
4. Using 'ensure' when describing responsibilities of software to differentiate outcomes in the scope of the code in question from side-effects of other elements of code.
5. Using mocks to stand-in for collaborating modules of code essentially stubbed out without the actual implementation.

e.g. RSPec, Cucumber

Selenium: Selenium IDE and Selenium RC. (Remote Control)

Selenium IDE provides automated way of recoding tests from a Firefox plug in and it has a powerful feature to validate tests by providing several commands to assert the results. It also provides a powerful mechanism to export the test case into a programming language such as Java, Ruby (RSpec),Perl, .etc.

Selenium RC: is a tool that allows you to launch browser sessions and run Selenium tests against those browsers.
To do this, the Selenium RC server acts as an intercepting proxy for the browser

Selenium-IDE does not directly support:

* condition statements
* iteration
* logging and reporting of test results
* error handling, particularly unexpected errors
* database testing
* test case grouping
* re-execution of failed tests
* test case dependency
* screenshot capture of test failures

Flash is not supported as a part of the Selenium-IDE but there is a flash extension for Selenium Flash and it also requires changing the actual Flash action scripts to add methods to be called from Selenium. It is not a very good practice.

The results from the tests can be verified by connecting from the database.

Watir: Watir is automated testing framework. It does not have a IDE of it's own but there is a testwide recorder that can be used. A combination of RSpec and Watir really makes very readable test scripts. It offers Flash support on firefox but I could not find a lot of documentation around this. So I have my doubt's around automating Flash components.

Cucumber is designed to allow you to execute automated integration tests. It is a tool for implementing Behavior Driven Development.It can work in Conjecture with Rspec.

Each feature is expressed as a Give\When\Then

Given: Preconditions to be filled.
When: This section defines what feature does (i.e., the behaviour, the steps).
Then: Testable out come that can be validated.


http://www.oreillynet.com/pub/a/ruby/2007/08/09/behavior-driven-development-using-ruby-part-1.html
http://www.ibm.com/developerworks/web/library/wa-rspec/
http://wiki.openqa.org/download/attachments/400/Selenium+IDE.swf?version=1
http://www.rubyinside.com/cucumber-the-latest-in-ruby-testing-1342.html

Character Encoding and Commerce

Character Encoding 101:

The "charset" parameter identifies a character encoding, which is a method of converting a sequence of bytes into a sequence of characters. This conversion fits naturally with the scheme of Web activity: servers send HTML documents to user agents as a stream of bytes; user agents interpret them as a sequence of characters. The conversion method can range from simple one-to-one correspondence to complex switching schemes or algorithms...

Reference: Section 5.2 Character encodings of the HTML Document Representation W3C Recommendations

Character encoding tells the browser and validator what set of characters to use when converting the bits to characters.

Types: what does this mean, there are different standards of character set encoding
You see various encoding types

Traditional ASCII is 7 bits and has limited character representation.
encoding="ISO-8859-1"
encoding="windows-1252
encoding="utf-8"

Even though UTF-8 is one of the most popular some of the old browsers might not support hence we used to use ISO-8859-1 encoding in some pages.

windows-1252 :
Windows-1252 is a character encoding of the Latin alphabet, used by default in the legacy components of Microsoft Windows in English and some other Western languages
ISO-8859-1 :
UTF-8,UTF-16,UTF-32 :
Unicode is an international character set standard which supports all of the major scripts of the world, as well as common technical symbols.
Unicode transformation unit and each character is store in 21-bits so depending on each of these UTF types it is stored as a multiple of 8 bytes
e.g. UTF-8 , it could be saved in upto 4 bytes.


Developers:

Java:
In the Java programming language char values represent Unicode characters. Unicode is a 16-bit character encoding that supports the world's major language
UTF-16 (Java).

In WCS:

The language table has a ENCODING column that specifies the encoding for language
e.g.

select encoding from language where language_id=-1;

From end in JSP:
Character encoding

We set the character encoding to UTF-8 as this is what is present in the DB for language encoding.

<%
response.setContentType("text/xml");
response.setCharacterEncoding("UTF-8");
response.setHeader("Cache-Control","max-age=0");
%>


Form Submit: In commerce what ever is specified in front end should match with what ever is specified in the encoding attribute of the language table.

select encoding from language where language_id= language passed in the command context.

Database:
e.g. If you need the character encoding to windows 1252, you can do that in the session while executing SQL statements.

set NLS_LANG=AMERICAN_AMERICA.WE8MSWIN1252

Database Client:

I use today and for the client to show the correct characterset, we have modified.

HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE\Key_OraClient...
Change “NLS_LANG” to AMERICAN_AMERICA.UTF8




Good reference URLs:
http://java.sun.com/javase/technologies/core/basic/intl/faq.jsp
http://java.sun.com/docs/books/tutorial/i18n/text/convertintro.html
http://tlt.its.psu.edu/suggestions/international/web/codehtml.html

JSP Error Handling and Extending OOB framework

Out of the box these 2 are good Articles:

http://publib.boulder.ibm.com/infocenter/wchelp/v6r0m0/topic/com.ibm.commerce.developer.doc/concepts/csdcommanderror.htm

http://publib.boulder.ibm.com/infocenter/wchelp/v6r0m0/topic/com.ibm.commerce.developer.doc/concepts/csdjsperror.htm

Essentially there are following elements:
1. ECSystemException and ECApplicationException classes, that are used to throw exceptions from inside the controller.
2. Extending the ECMessage and ECMessageKey
3. The errors are intercepted by the ExtStoreErrorDataBean and mapped to resource bundles
4. Errors are displayed in JSP.

Define the property in ExtMessages_en_CA.properties (
_ERR_RESTRICTED_PROD = This item is restricted for delivery.


Extending ECMessage and ECMessageKey:

public interface EXTMessageKey extends ECMessageKey{
public static final String _ERR_RESTRICTED_PROD = "_ERR_RESTRICTED_PROD";

public static final String _ERR_MAX_NUM_OF_OF_CARTS_REACHED = "_ERR_MAX_NUM_OF_OF_CARTS_REACHED";

}

public class EXTMessage extends ECMessage{
//this file structure needs to be created inside toolkit properties folder
for com/ext/common/messages/EXTMessages.propertie

public final static String EXT_MESSAGES_RESOURCE_BUNDLE = "com.ext.common.messages.EXTMessages";

// final constants
public static final int USER = ECMessageType.USER;
public static final int SYSTEM = ECMessageType.SYSTEM;

// message severities
public static final long ERROR = ECMessageSeverity.ERROR;
public static final long WARNING = ECMessageSeverity.WARNING;
public static final long STATUS = ECMessageSeverity.STATUS;
public static final long INFO = ECMessageSeverity.INFO;

public EXTMessage(long msgSeverity, int msgType, String msgKey)
{
super(msgSeverity, msgType, msgKey, EXT_MESSAGES_RESOURCE_BUNDLE);
}

//system
public static final EXTMessage _ERR_MAX_NUM_OF_OF_CARTS_REACHED = new EXTMessage(ERROR, SYSTEM, EXTMessageKey._ERR_MAX_NUM_OF_OF_CARTS_REACHED);

public static final EXTMessage _ERR_PROCESSING_REQUEST = new EXTMessage(ERROR, SYSTEM, EXTMessageKey._ERR_PROCESSING_REQUEST);

//user
public static final EXTMessage _ERR_LOGIDPWDPSID_NOT_PRESENT = new EXTMessage(ERROR, USER, EXTMessageKey._ERR_LOGIDPWDPSID_NOT_PRESENT);

public static final EXTMessage _ERR_RESTRICTED_PROD = new EXTMessage(ERROR, USER, EXTMessageKey._ERR_RESTRICTED_PROD);
}


In the controller command, The error needs to be thrown as an exception for example:

throw new ExtendedApplicationException( EXTMessage._ERR_RESTRICTED_PROD,
CLASSNAME,
METHOD_NAME,
ORDER_SHOP_CART_VIEW,
true);

Commerce Developer : Working with ORACLE

As a commerce developer working with Oracle back end, it is very important to have good understanding of SQL.

I am not a DBA, but the way I have learned was to started with basic Insert/Update/Delete, Select commands and using group by, order by
but progressively learned some handy SQL commands.

Refresh local schema for development.
I would use the script below for creating tablespace/schema/assign grants and import schema. Replace wcsatdev with the schema name of your choice.

drop tablespace atdata including contents and datafiles;
drop tablespace atindx including contents and datafiles;

create tablespace atdata
datafile 'C:\oracle\product\ORCL10G\atdata_01.dbf' size 250m reuse autoextend on;

create tablespace atindx
datafile 'C:\oracle\product\ORCL10G\atindx_01.dbf' size 250m reuse autoextend on;

drop user wcsatdev cascade;

create user wcsatdev identified by wcsatdev
default tablespace atdata
temporary tablespace temp
quota unlimited on atdata
quota unlimited on atindx;

grant connect,resource,create materialized view to wcsatdev;

grant create view to wcsatdev;

grant create synonym to wcsatdev;

create or replace directory dpdumpdir as 'C:\projects\db';

create or replace directory dplogdir as 'C:\projects\db';

GRANT READ, WRITE ON DIRECTORY DPDUMPDIR TO wcsatdev;

GRANT READ, WRITE ON DIRECTORY DPLOGDIR TO wcsatdev;

Using data pump in Oracle 10g, faster way to import.
impdp system/oracle@orcl10g dumpfile=DPDUMPDIR:export.dmp logfile=DPLOGDIR:from_wcsatdev_10g.log REMAP_SCHEMA=from_schema:to_schema PARALLEL=8 CONTENT=ALL



When ever you have a larger database on your local database, it is a good idea to run DB stats once in a while to improve performance.

exec dbms_stats.GATHER_SCHEMA_STATS(ownname=>'WCSATDEV',estimate_percent=>dbms_stats.auto_sample_size, CASCADE=>TRUE, DEGREE=>4)

Where degree will invoke the 4 parallel slaves, cascade is required for indexes.

Size of the Tables: Run this command in the schema

select SEGMENT_NAME,sum(BYTES)/(1024) size_in_kil from user_extents where segment_type='TABLE' group by SEGMENT_NAME order by size_in_kil desc;


Dropping Stage prop Triggers: Stage prop is a completely new discussion but on my local I run this for better performance.

set pages 0 lines 100 ;
spool drop_trigger1.sql ;
select 'drop trigger '||trigger_name||' ;' from user_triggers where trigger_name like '%STG_%' or trigger_name = 'STGLOG_STGRESERVED1' ;
spool off;
@drop_trigger1.sql;



Issues and Fixes:

java.sql.SQLException: ORA-01000: maximum open cursors exceeded

check for this. select count(*) from v$open_cursor;

Login with system previleges and type show parameter OPEN_CURSOR

Alter cursors
ALTER SYSTEM SET OPEN_CURSORS=4000 SCOPE=BOTH;

Soln: restart DB


Error: ORA-12514: TNS: listener does not currently know of service requested in connect descriptor.

I made these changes and since then i did not see the problem.
ORACLE_HOME\network\admin\listener.ora, Please look for HOST and change that to
HOST=Windows hostname
ORACLE_HOME\network\admin\tnsnames.ora
For the 10g local instance tnsentry: give localhost instead of the windows hostname
HOST=localhost

Debug: Generic System Error Test JSP

This would happen due to the descripency between supported languages in wc-server.xml and language table.

_ERR_WRITE_JSP

EJB Files to be Checked Into CVS

Creating an EJB bean and best practices to checkin files into CVS for 6.0.

The most important rule, while creating EJB's is to make sure, you understand the data model and finding the composite key that would identify the table.
e.g. extending catentry_ext (catentry_id,store_id), There should be just 1 ejbcreate with both the values.
e.g. extending catgroupdesc_ext (catgroup_id, language_id), ejbcreate(catgroupId,langId)
Also making sure the finders exist with the composite key.
e.g. findByCatentryIdStoreId and findByCatGroupIdAndLangId


I have used the following link to create multiple EJB's successfully.

http://publib.boulder.ibm.com/infocenter/wchelp/v6r0m0/topic/com.ibm.commerce.developer.tutorial.doc/tutorial/ttd18.htm?resultof=%22%62%6f%6e%75%73%22%20%22%62%6f%6e%75%22%20%22%62%65%61%6e%22%20

2. I would usually be careful at step 15. to map the entity bean to the table, some times has issues when you try to map in wrong view. Other than that all other steps are required.

3. Best practice to check in files as a practice. Usually the following files are checked in and an ant task is used during build to generate all the stub classes for EJB's and access beans. It is better to not checkin all the generated classes to CVS. This approach would make it cleaner.
packagename.XXCatEntryExtBean
packagename.XXCatEntryExtKey
packagename.XXCatEntryExtHome
packagename.XXCatEntryExt

WebSphereCommerceServerExtensionsData\ejbModule\META-INF\ejb-jar
WebSphereCommerceServerExtensionsData\ejbModule\META-INF\ibm-ejb-access-bean.xmi
WebSphereCommerceServerExtensionsData\ejbModule\META-INF\ibm-ejb-jar-bnd.xmi
WebSphereCommerceServerExtensionsData\ejbModule\META-INF\ibm-ejb-jar-ext.xmi

META-INF\backends\ORACLE_V9_1\Map.mapxmi
META-INF\backends\ORACLE_V9_1\WEBSPHERECOMMERCESERVEREXTENSIONSDATA_NULLID_XX_CATENTRY_EXT.tblxmi

4.Do not check-in access beans, use a build script to create access beans, when refreshing code from version control system such as CVS or SVN.
To test it, you can use access bean to create a new row and commit.

Interesting things. Why do you use access beans?
Access beans are wrappers around EJB's that providers getters and setters to the fields and are also used to improve the performance of an data update operations, where in multiple columns are written to first Hashmap internally and committed to the database.

TrashCan User in WCS

Trash Can Users And the mighty fear of loading a junkcarts in Cart.

If the default trash can user is enabled, it would not remove the row from the database directly, but hold in the database with a state 'J', when ever a merge cart or orderitem/requisition is deleted. This is usually done for performance improvement since in a live site when trying to delete order items would result in database issues.

This is configured in wc-server.xml

TrashCanEnabled="true"/>

When this enabled, these orders are not deleted and are moved to a state 'J' (Junk Cart)

-1002 in commerce is the generic user.

In one of the commerce sites that I worked, there were huge performance concerns due to the fact that a certain scenario could load junk carts for user -1002.
We modified the trashcan to use a custom user.