Tuesday, December 4, 2012

Access control for Views

The data load project in V7 makes, it really easy to load the access control changes. This blog explains a sample custom view access policies for guest and registered user and how this can be verified in the database using the query below. Even for guest users to access views, it is required for the view to belong to AllSiteUsersViews policy. If one tries to create a view and  access it, you would get the following error below and -1002 is (generic user). The first time user comes on a site, it would run user as generic user and as soon as it tries to perform any action specific to a user such as add to cart, it is converted to a guest user.

Error without policy:
AccManager isAllowed CMN1501E: User -1002 does not have the authority to perform action "NewView" on resource "com.ibm.commerce.command.HttpForwardViewCommandImpl" for command "null".

view_acpolicies.xml

<!DOCTYPE Policies SYSTEM "../dtd/accesscontrolpolicies.dtd">
<Policies>
<Action Name="GuestView" CommandName="GuestView" />
        <Action Name="RegisteredView" CommandName="RegisteredView" />
   <!-- Guest USERS VIEWS -->
<ActionGroup Name="AllSiteUsersViews" OwnerID="RootOrganization">
<ActionGroupAction Name="GuestView" />
</ActionGroup>

  <!-- REGISTERED USERS VIEWS -->
<ActionGroup Name="RegisteredUserViews" OwnerID="RootOrganization">
<ActionGroupAction Name="RegisteredView" />
  </ActionGroup>

</Policies>
-------------------------------------------------
select * from acpolicy where acactgrp_id in (select acactgrp_id
from acactactgp where acaction_id
in (select ACACTION_ID from ACACTION where ACTION = 'GuestView'))

Friday, November 9, 2012

SOLR adding custom attributes in SRCHATTRPROP !


If you need to filter SOLR queries based on a dynamic attribute. In the sample java code below the value for the custom attribute _custom.ItemType is retrieved from SRCHATTRPROP table. The code uses out of the box getIndexColumnNameByType on SolrSearchConfigurationRegistry object to retrieve the custom attribute value.

Monday, November 5, 2012

Elite Store Order Flow JSPs

Out of the box V7->EliteStorefrontAssetStore ->Order flow.
Useful to check for JSP's in a certain flow quickly. I have moved header\footer\mincart\breadcrumb\leftside\rightside\espot JSP's to the bottom.Internally it calls multiple JSP fragments that are not captured here.

Cart --> ViewName->AjaxOrderItemDisplayView
Starting point of flow JSP::OrderItemDisplay.jsp
/EliteStorefrontAssetStore/ShoppingArea/ShopcartSection/OrderItemDisplay.jsp
/EliteStorefrontAssetStore/ShoppingArea/OrderCreationSection/RequisitionListSubsection/RequisitionListPopupDisplay.jsp
/EliteStorefrontAssetStore/ShoppingArea/OrderCreationSection/RequisitionListSubsection/RequisitionListPopupTableDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/Promotions/PromotionChoiceOfFreeGiftsPopupContent.jsp
/EliteStorefrontAssetStore/ShoppingArea/ShopcartSection/ShopCartDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Order/Cart/ShipmodeSelectionExt.jsp
/EliteStorefrontAssetStore/Snippets/Order/Cart/OrderItemDetail.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/Promotions/PromotionPickYourFreeGift.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/SingleShipmentOrderTotalsSummary.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/Promotions/PromotionCodeDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/Promotions/CouponWalletTable.jsp
/EliteStorefrontAssetStore/Snippets/Order/Cart/CheckoutLogon.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedHeaderDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/ESpot/CachedContentAreaESpot.jsp
/EliteStorefrontAssetStore/include/MiniShopCartDisplay.jsp
/EliteStorefrontAssetStore/include/BreadCrumbTrailDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedLeftSidebarDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedRightSidebarDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/ESpot/ESpotNavDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedFooterDisplay.jsp

Shipping and Billing page--> OrderShippingBillingView
Starting point of flow JSP:OrderShippingBillingDetails.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/OrderShippingBillingDetails.jsp
/EliteStorefrontAssetStore/ShoppingArea/OrderCreationSection/RequisitionListSubsection/RequisitionListPopupDisplay.jsp
/EliteStorefrontAssetStore/ShoppingArea/OrderCreationSection/RequisitionListSubsection/RequisitionListPopupTableDisplay.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/ShippingDetailsDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/Promotions/PromotionChoiceOfFreeGiftsPopupContent.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/ShippingAddressSelect.jsp
/EliteStorefrontAssetStore/Snippets/Member/Address/AddressDisplay.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/SingleShipmentShippingMethodDetails.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/SingleShipmentShipChargeExt.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/OrderItemDetails.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/Promotions/PromotionPickYourFreeGift.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/SingleShipmentOrderTotalsSummary.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/Promotions/PromotionCodeDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/Promotions/CouponWalletTable.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/ScheduleOrderDisplayExt.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/BillingAddressDropDownDisplay.jsp
/EliteStorefrontAssetStore/Snippets/EDP/PaymentMethods/Empty.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedHeaderDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/ESpot/CachedContentAreaESpot.jsp
/EliteStorefrontAssetStore/include/MiniShopCartDisplay.jsp
/EliteStorefrontAssetStore/include/BreadCrumbTrailDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedLeftSidebarDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedRightSidebarDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedFooterDisplay.jsp

 Order Summary page-->SingleShipmentOrderSummaryView
 Starting point of flow JSP:BillingAddressDropDownDisplay
 /EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/BillingAddressDropDownDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Member/Address/AddressDisplay.jsp
/EliteStorefrontAssetStore/Snippets/EDP/PaymentMethods/StandardCOD.jsp
/AjaxActionResponse.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/OrderTotalInJSON.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/SingleShipmentSummary.jsp
/EliteStorefrontAssetStore/include/MiniShopCartDisplay.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/OrderItemDetailSummary.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/SingleShipmentOrderTotalsSummary.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/ScheduleOrderDisplayExt.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedHeaderDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/ESpot/CachedContentAreaESpot.jsp
/EliteStorefrontAssetStore/include/BreadCrumbTrailDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedLeftSidebarDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedRightSidebarDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedFooterDisplay.jsp

Order Confirmation page-->OrderShippingBillingConfirmationView
Starting point of flow JSP:OrderReceivedNotify.jsp
/EliteStorefrontAssetStore/Messages/OrderReceivedNotify.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/Content/ContentSpotDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Member/Address/AddressDisplay.jsp
/AjaxActionResponse.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/OrderShippingBillingConfirmationPage.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/OrderItemDetailSummary.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/SingleShipment/SingleShipmentOrderTotalsSummary.jsp
/EliteStorefrontAssetStore/ShoppingArea/CheckoutSection/ScheduleOrderDisplayExt.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedHeaderDisplay.jsp
/EliteStorefrontAssetStore/Snippets/Marketing/ESpot/CachedContentAreaESpot.jsp
/EliteStorefrontAssetStore/include/MiniShopCartDisplay.jsp
/EliteStorefrontAssetStore/include/BreadCrumbTrailDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedLeftSidebarDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedRightSidebarDisplay.jsp
/EliteStorefrontAssetStore/include/styles/style1/CachedFooterDisplay.jsp

Thursday, November 1, 2012

Increase HeapSize on server.xml for OutOfMemory Exception

If you are running SOLR or any other high memory process, there is a chance, you have seen the OutOfMemory exception. You can increase the heap-size using WAS or directly in the file. Recommended approach would be directly in the file. 

From WAS console:
  1. In the administrative console, click Servers > Server Types > WebSphere application servers, and then click on an application server name.
  2. In the Server Infrastructure section, click Java and process management > Process definition .
Directly updating the file, as this is the same file that gets updated while changing WAS console: On toolkit, this would be the path
C:\WCDE_ENT70\wasprofile\config\cells\localhost\nodes\localhost\servers\server1\server.xml
Search for initialHeapSize and maximumHeapSize and change 
initialHeapSize=512
maximumHeapSize=1536
or you have only 1024 try
initialHeapSize=1024
maximumHeapSize=1024
If you give incorrect values here, it could result in your server not starting so please be cognizant about that.

Analyzing memory heaps from WebSphere Commerce


You need two install 2 plug-ins in RAD:
  1. Eclipse Memory Analyzer Plug-in: http://www.eclipse.org/mat/
  2. IBM Diagnostic Tool Framework for Java Plug-in: http://www.ibm.com/developerworks/java/jdk/tools/dtfj.html#download

This should allow you to read WCS dump files (*.phd). Install #1 then #2.

After that you can follow this tutorial to get ideas on what to investigate:

Friday, September 21, 2012

Activating users using a script

This is usually a problem with a lot of sites where due to a security password policy, users are deactivated after a certain number of unsuccessful attempts in trying to log in. It could be frustrating for support folks to manually activate the users.

A cron job can be created with a script to activate users deactivated in the last day. There is 2 things required one to run the script below and second to refresh the customer profile cache registry.

Step1:

update userreg
set status=1,passwordinvalid=null
where trunc(passwordinvalid) >= sysdate-1 and status=0;

Step2:
Refresh Customer profile Cache Registry

Tuesday, September 18, 2012

Installing RAD SQL Profiler in WebSphere Commerce Developer 7

The IBM SQL profiler is a great tool to find database related performance problems.
However, the current guide only covers how to install it on WCS 6.

Here is an updated list of steps for those of you that need to install it in WCS developer 7.

Sunday, September 16, 2012

Dynacache Tip: Dynamically skip cache for a page

The usual way most people diagnose DynaCache problems is by commenting out or removing the cache-entry or cache-id snippet in cachespec.xml. Since cachespec.xml changes are loaded without restarting, this works pretty well.

While this works fine, sometimes I want to be able to see what the page looks like without caching quickly, without having to edit cachespec.xml. Sometimes the changes to cachespec.xml would require creating a support ticket to modify a QA or DEV environment.

A more dynamic solution is to use a servlet filter to disable caching for a top level page without touching DynaCache.

Here is code for such a servlet filter:


This filter will disable caching when the URL contains the parameter "nocache" with value "true".
Example: http://localhost/webapp/wcs/stores/servlet/StoreCatalogDisplay?langId=-1&storeId=10151&catalogId=10551&nocache=true

Keep in mind that this only disables caching for the URL you are hitting in your browser and does not affect caching of child fragments which are not consumed by that URL.

This filter should only be used in DEV or QA environments.

Wednesday, September 5, 2012

Dynacache Tip: Exclude JSP fragment from cache

Scenario:

1. You are caching a JSP in cachespec.xml, lets call it "parent.jsp".
2. You have configured "parent.jsp" to consume sub-fragments.
3. There is a JSP being included by parent.jsp, lets call it "child.jsp"
4. The file is being included using the <c:import> tag
5. You don't want to cache child.jsp at all
6. You don't want parent.jsp to consume child.jsp into its cache

Solution:

Add the following line in child.jsp:

<% ((com.ibm.websphere.servlet.cache.ServletCacheResponse)response).setDoNotConsume(true); %>

This way gets around having to add child.jsp to cachespec.xml and setting do-not-consume=true.

Thursday, August 23, 2012

Error invalid LOC header (bad signature) | jar binary


The Jar files custom or out of the box is important to be checked in as binary file into the version control system. If you check them into CVS or SVN as text files. It usually not a problem if your build process is on windows as windows is more forgiving about the file types checked in. If you use Linux or any UNIX based CVS\SVN servers. The build would through up with the error as below. The fix is either to override and check in the jar files as binary files or run some commands to change the type of the JAR files in CVS.


this is the error i am getting
[accessBeanRegeneration] +++ Warning +++:  java.lang.InternalError: jzentry == 0,
[accessBeanRegeneration]      jzfile = 164252568,
[accessBeanRegeneration]      total = 5,
[accessBeanRegeneration]      name = C:\WCToolkitEE60\workspace\WC\lib\custom\trimflt.jar,
[accessBeanRegeneration]      i = 1,
[accessBeanRegeneration]      message = invalid LOC header (bad signature)
[accessBeanRegeneration]      at java.util.zip.ZipFile$2.nextElement(ZipFile.java(Compiled Code))
[accessBeanRegeneration]      at org.eclipse.jdt.internal.core.JarPackageFragmentRoot.computeChildren(JarPackageFragmentRoot.java(Compiled Code))
[accessBeanRegeneration]      at org.eclipse.jdt.internal.core.PackageFragmentRoot.buildStructure(PackageFragmentRoot.java(Compiled Code))
[accessBeanRegeneration]      at org.eclipse.jdt.internal.core.Openable.generateInfos(Openable.java(Compiled Code))
[accessBeanRegeneration]      at org.eclipse.jdt.internal.core.JavaElement.openWhenClosed(JavaElement.java(Compiled Code))

Tuesday, August 7, 2012

When is this file created ***Factory.java during the EJB creation process?

Answer after the access bean is created. If you haven't created entity bean for a while like me and you could almost forget. If you are looking at creating access bean for a session bean. It does not work with copy helper and you have to select the Java bean wrapper. After the access bean is created for session bean using New wizard, and select EJB > Access Bean and click Next.


Select the appropriate bean type for the session bean in this case would be Java bean wrapper.



Once you are done with the wizard, it creates the ****AccessBean.java and ***Factory.java and ibm-ejb-access-bean.xmi is updated.
Sample block from ibm-ejb-access-bean.xmi

If you just need to regenerate access beans, you can do the following and then deploy.

Java EE->AccessBeans --> Regenerate access beans
Java EE->Prepare for Deployment.


Just want to also quickly touch upon why access beans are required?
Access beans provide a simpler interface to clients, caching of the home object, and reduced call traffic to the enterprise bean and also encapsulates all the complex logic.
At run time the access bean caches the enterprise bean home object because look ups to the home object are expensive, in terms of time and resource usage.
The access bean implements a copyHelper object that reduces the number of calls to the enterprise bean when commands get and set enterprise bean attributes. Only a single call to the enterprise bean is required when reading or writing multiple enterprise bean attributes.

Reference:
http://publib.boulder.ibm.com/infocenter/wchelp/v6r0m0/topic/com.ibm.commerce.developer.doc/concepts/csdaccessbeans.htm

Monday, July 30, 2012

Websphere commerce--ERP data flow !!

For a lot of front end developers not working on integration, the ERP system is like a Black hole. This blog presents high level flow and integration touch points. As a part of most B2B and B2C implementation, it is generally important to communicate with a back end ERP system. 
The core of order management system in ERP has at least 3 main documents, Sales Order Doc, Delivery Doc, Invoice Doc. Sales Doc is used to create orders, delivery doc involves with shipping, Invoice with PO and settlement.
There could several other feeds and integration's depending on the operations in the end ERP system but most e-commerce system are fed Product\Pricing\Inventory data from back-end.  All this data being transported between the systems, could be implemented as Feeds\Web services\JMS-MQ .
Enterprise middle-ware systems such as Web Methods\WebSphere ESB\Open Source ESB could be for data transformations and communication with multiple systems.


Products\Price\Inventory--core of any commerce site and if it is driven from your ERP system, here's how the flow would be:


If the orders are dropped into the ERP system for order fulfillment at the high level, order tunnel flow from commerce to the backend system.





Tuesday, June 26, 2012

Logon failure Issue | URLRedirect filter enabled


If your user logon flow is working fine in development environment and is failing once you get to staging or live. Essentially there is a configuration that needs to be defined in wc-server.xml for allowedDomain if the URLRedirectFilter is enabled.

The URL redirectfilter is enabled to prevent phishing attacks where a link from the current site would prevent the user from being redirected to a different site. It does need correct allowedDomain or else some of the functionality including logon starts failing.

Make sure in wc-server.xml in store definition has correct value  to domain or sub-domain value in the URLredirectfilter definition for store element.

alloweddomain name="example.com" 

Logs from Error for this scenario with login failure:

[6/22/12 12:34::024 EDT] 00000123 EJBMDOrchestr A   CNTR0117I: Container-managed ejbStore() call will be bypassed on bean: "WC_ejgrp#Member-MemberManagementData.jar#UserPasswordHistory" if it has not been modified in the current transaction.
[6/22/12 12:34:58:313 EDT] 0000023b CommerceSrvr  A ECActionForwardInstance isRedirectAllowed(String) CMN0207E The value of the parameter "URL" is not correct.
[6/22/12 12:59:59:254 EDT] 00000113 CommerceSrvr  E WCAuthenticationCookie getUserId CMN1039E: An invalid cookie was received for the user, your logonId may be in use by another user.
[6/22/12 12:34:28:727 EDT] 00000122 CommerceSrvr  A ECActionForwardInstance isRedirectAllowed(String) CMN0207E The value of the parameter "URL" is not correct.
[6/22/12 12:34:30:239 EDT] 0000045b CommerceSrvr  E WCAuthenticationCookie getUserId CMN1039E: An invalid cookie was received for the user, your logonId may be in use by another user.
[6/22/12 12:34:13:330 EDT] 00000245 CommerceSrvr  A ECActionForwardInstance isRedirectAllowed(String) CMN0207E The value of the parameter "URL" is not correct.
[6/22/12 12:34:15:141 EDT] 00000245 CommerceSrvr  E WCAuthenticationCookie getUserId CMN1039E: An invalid cookie was received for the user, your logonId may be in use by another user.

Tuesday, June 19, 2012

IBM Management Center | Preview not working

When you click on the preview functionality from the management center and it fails with the following error

[5/8/12 15:43:15:374 CDT] 00000026 SystemOut O ErrorDataBean:
Originating command:
Message Type: 0
Message Key: _ERR_REDIRECT_VIEW_ERROR
Message: CMN3101E The system is unavailable due to "URL".
[5/8/12 15:47:28:845 CDT] 0000002c CommerceSrvr E WCAuthenticationCookie getUserId CMN1039E: An invalid cookie was received
for the user, your logonId may be in use by another user.
[5/8/12 15:47:28:856 CDT] 0000002c SystemOut O ErrorDataBean:
Originating command:
Message Type: 1
Message Key: _ERR_INVALID_COOKIE
Message: CMN3101E The system is unavailable due to "{0}".
[5/8/12 15:49:06:561 CDT] 00000027 SystemOut O ErrorDataBean:
Originating command:
Message Type: 0
Message Key: _ERR_REDIRECT_VIEW_ERROR
Message: CMN3101E The system is unavailable due to "URL".

In wc-server.xml under respective WebModule all webserver hostname and domainname are added as AllowedHost

Monday, May 14, 2012

Cookie path creation always at the web server root !!

If you use the default settings on cookie, it creates the cookies at the web server context. In WebSphere commerce that would create the cookies at /webapp/wcs/stores/servlet with default settings. This is usually not a problem for most scenarios but if you have a scenario when there is a canonical URL e.g for SEO,  that does not start with /webapp/wcs/stores/servlet. The HTTP response stops carrying the cookies to the browser and you will start seeing issues in the site where you have code that depends on the cookie.


Sample code to create a custom cookie at the root:
String value="custom_cookie_value";
Cookie customCookie = new Cookie("CustomCookie",value);
customCookie.setPath("/");
Even though this is a simple thing to fix, it could be hard to debug. As a best practice, always create custom cookies at the root and out of the box cookies are always created at the root.


Primer on cookies:
A cookie is a application specific data stored as a separate file for each domain in the browser. There are 2 types, session cookies and persistent cookies. Session cookies are used to log information that are eventually erased after the life cycle of a session, e.g Based on a users Login, if you want to show certain attributes or mini-carts and are deleted on LogOff.
Persistent cookies are store for much longer period and are useful for things like RememberMe functionality.


Validation rules to pass the cookies back to web-server:
A cookie is sent back from browser to web-server based on certain events when it matches the Domain, Path and port of the server in cookie.
Cookie can have a secure attribute set if it is on HTTPS.
Each cookie has a Max-Age and the browser would not send back expired cookies.

Tuesday, May 8, 2012

Bazaar Voice Integration | Reviews and Ratings

BazaarVoice provides a really good platform for product feedback and social media integration's. It is really easy for business users and programmers to integrate with back end systems.

Essentially BazaarVoice Integration has the following main components:

1. Display reviews to guest and registered customers.
2. Writing new reviews and as a platform providing business users access to their workbench tooling to approve reviews.
3. This is optional and provides SEO benefits as the JS based integration makes it harder for search engines to crawl the content and hence BazaarVoice provides SEO basic and SEO advanced integration's, this is a server side include of product files.
4. Sending outbound XML product feed to BazaarVoice nightly or weekly feed of categories and products, depending on your product cycle.
5. Feeding reviews and ratings file from BazaarVoice back to your Search engine (Endeca or SOLR).  This is to improve the search on the site by allowing keywords from reviews to be searchable.
Find below sample code for a simple integration piece to show reviews.




Resources: Great place with examples.
https://github.com/bazaarvoice/HostedUIResources

Sunday, April 29, 2012

Adding cache-control on static assets | Improve data integrity and performance


Static files (example: JS\CSS\HTML.) are cached by client browsers and if they don't carry cache-control directive in the HTTP-header, it is up to the browsers to determine, how often to check to check for updated value from server. Some times browsers could cache stale content for days.
This could cause the static file to be obsolete and pick from local browser cache and out of sync and could potentially result in broken pages and corrupt files.
The solution is to add cache-control directive to the web server configuration and this will force the browser to check for the updated version from the server when the time specified expires.
The functionality would essentially result in HTTP_STATUS code 200 when a file is modified and other wise would return HTTP_STATUS code 304

Testing the HTTP-Header changes:
You can using tamper data as Firefox plugin or fiddler for IE to look at the HTTP-header

Make changes to httpd.conf
1. Add the following at the end of LoadModule section in web server configuration file (httpd.conf).
        LoadModule headers_module modules/mod_headers.so
2. Add the following at the end of web server configuration file (httpd.conf).


Scenario1: [Browser cache not expired and file changed - File should be fetched with 200 response]
Scenario2: [Browser cache expired and file changed - should be fetched from server with 200 response]
Scenario3: [File not changed but browser cache expired - should get 304 response]
Scenario4:[File not changed and browser cache not expired - should be serverd from browser cache]

Friday, April 27, 2012

Sorting !!! an important function for a lot of display pages

This blog is for sorting using Java code. Sorting is an essential part, which is a required function in most code bases.I am going to start with a couple and then will add few more. If you any of you guys reading this have better ideas, please add in the comments.

//Sort a string with a delimiter and arrange with the same delimiter into a string Arrays.sortOf() uses merge sort and has a worst case performance O(nlogn)
//Sort and maintain the same ordering. LinkedHashMap is very useful to preserve the ordering. It Maintains a doubly-linked list running through all of it's entries

Saturday, March 31, 2012

Introduction to WCS Cache Invalidation - Part 1

"There are only two hard problems in Computer Science: cache invalidation and naming things."
-- Phil Karlton

As the quote above suggests cache invalidation can be a complex topic.
In this post will discuss the basics of cache invalidation; along with some guidelines for the different invalidation strategies that can be used with WebSphere Commerce.
This guide assumes some familiarity with configuring caching in WebSphere.

What is cache invalidation ?
Essentially it is the mechanism of removing the stale items from cache.
WebSphere Commerce has a caching system called DynaCache.

Why is invalidation important?
To protect the accuracy of the data available in cache. 
Most business applications have a consistency requirement for certain types of data.
For example, If my inventory reaches zero, I don't want the customer to purchase items which have no inventory and which can not be back-ordered.

How can I invalidate cache?
Here is a list of the different ways you can invalidate cache:
  1. Manual Invalidation
  2. Time to Live (TTL)
  3. Dependency IDs
    1. Using command invalidation
    2. Using CACHEIVL (only in WebSphere Commerce)
    3. Using custom code
Manual Invalidation
This type of invalidation is done by using the WebSphere Commerce Cache Monitor.
The cache monitor is a web application which uses the DynaCache API to remove specific entries.

The EAR file comes with WebSphere Application Server but it is not installed by default.
Make sure servlet caching is enabled in your WCS environment before installing the Cache Monitor.

The advantage of this method is that you can invalidate all entries at the same time by pushing a button.
You can also invalidate only a given group of entries. This is a more complex topic which will be explained in a later post.

The disadvantage is that if you have a large number of entries it will be difficult to find a specific entry.
There is no search capability and the navigation is not very user friendly.

Invalidation using a TTL
The TTL value is configured in cachespec.xml.
A <timeout> child element is added to a <cache-entry> element. The timeout is specified in seconds.
This type of invalidation is best used when you don't have a hard consistency/freshness requirement.

Let's use the caching of the product display page as an example
For this example we will assume the following:
  1. The product table is updated once a day
  2. The update is done by a scheduled job
  3. The scheduled job runs once a day at 10pm
In this case it would make sense to set the TTL for ProductDisplay to 86400 (seconds in a day):



The timeout starts from the time each cache entry is created. Each product has a separate cache entry which is created when a page for a specific product id or part number is accessed. Since each cache entry could be created at a different time, they could also expire at a different time.

The advantage of this method is that it is easier to configure compared to dependency ID invalidation.

The main disadvantage of using TTL invalidation is that there is no guarantee that the cache will expire before the product table is updated.

Here is a scenario where the cache would not be properly invalidated:
  1. Most cache entries expire around midnight.
  2. The scheduled job is delayed due to some problem.
  3. A user access the product page for part number ABC at 12:30am creating a new cache entry
  4. The scheduled job updates the data for ABC and finishes running at 1am
  5. The product page for ABC is not updated because the timeout value was reset at 12:30am

Ideally the product update job should always run before the cache expires, but in this case it didn't.
The easiest solution in this case is to do a manual cache invalidation at 1am after the scheduled job has finished running.

Another disadvantage is that all the entries will expire around the same time. This will cause a large number of requests to hit WebSphere Commerce at the same time without the benefit of a cached response. This will degrade the performance perceived by the user.

In conclusion this method is best used when:
  1. You have a low consistency business requirement: The cache can be updated sometime after the database is updated.
  2. You have the operational resources: Someone is watching the scheduled job to see if it is delayed and if something goes wrong he knows to manually invalidate the cache.

NOTE: The <timeout> value is separate and different from the <inactivity> value. It is possible to have both settings. The <inactivity> value will expire the cache after it has not been accessed for a given number of seconds.

Stay tuned for the next post which will cover dependency ID based cache invalidation ...

Wednesday, March 28, 2012

A Classic case of redirect and passing objects !!

Redirect does not support passing objects back to view. I am sure, we have all tried to pass an object or ArrayList of objects to make a pretty URL in the browser or convert a POST request to a GET and so instead of doing a forward, we try to do redirect  from a command\controller to a view.

Controller command:
In JSP:

Error:
Unable to find a value for "methodName" in object of class "java.lang.String" using operator "."

HTTP redirection allows servers to redirect a client request to a different location. A POST request can be a converted to a GET request using redirection with a status code 302.A server specifies redirection by sending 3XX status code.
Redirect is done via  URL and hence can only support name/value pair. Forward works with Objects but redirects can not pass objects back to the view. When trying to access the object retrieved from WCParam, it throws the above exception.

Alternate Solutions: If you do need to pass an object on redirect.
1. Putting object in HTTPSession (Not recommended as it could use a ton of memory in application server depending on how many users hit the URL)
2. Serialize objects in a XML/JSON String, like what Accelerator does.
3. Commerce context services, customize a commerce service for storing session information and store in database.


Thursday, March 8, 2012

Demonstrating an Industry Standard Search Solution


Demonstrating an Industry Standard Search Solution

IBM WebSphere Commerce is now a fully operational search engine.
The product now includes Lucene Solr (http://lucene.apache.org/solr/) as the internal search engine, thus not only leveraging open standards, also incorporates industry best practices.

With that, you will probably benefit to enable the IBM WebSphere Commerce search capability in a starter store, so that you can get a good idea what is available out-of-the-box with the product.

Here is a link to the IBM WebSphere Commerce information center documentation regarding WebSphere Commerce search:
http://publib.boulder.ibm.com/infocenter/wchelp/v7r0m0/topic/com.ibm.commerce.developer.doc/concepts/csdsearch.htm

At a high level, WebSphere Commerce search provides the following key business benefits:

  • It is built on the top of open architecture.
  • It contains a rich set of search functionality for shoppers in starter stores.
  • It provides integrated search management tooling for business users in the Management Center.
  • It lowers the total cost of deployment and ownership, since its functionality is included as a feature of WebSphere Commerce.


In regards to starter stores, here is a link to the IBM WebSphere Commerce information center to summarize what is available-to-market in the product:
http://publib.boulder.ibm.com/infocenter/wchelp/v7r0m0/topic/com.ibm.commerce.starterstores.doc/concepts/csmsearch.htm

This blog entry covers the steps required to enable search for a starter store.

This is where you should start to understand how to setup search for a starer store:
http://publib.boulder.ibm.com/infocenter/wchelp/v7r0m0/topic/com.ibm.commerce.starterstores.doc/refs/rsdfepfeatroadmap.htm

As a side note to this link, I was only able to install Feature Pack 2, due to limited resource access, but I still brought IBM WebSphere Commerce up to Fix Pack level 5 (WCS v7.0.0.5).

The following list of steps is the procedure that I took in order to enable a fully integrated search solution into a demonstrable starter store:

  1. Install "WebSphere Commerce Developer Enterprise" version 7
  2. Update "WebSphere Commerce Developer Enterprise" to version 7.0.0.5
  3. Install "Feature Pack 2" (preferably feature pack 3 or higher)
  4. Enable the Store Enhancement Features
    1. cd 'WCDE_installdir'\bin
    2. enableFeature.bat -DfeatureName=store-enhancements
  5. Publish the Madisons Feature Pack Store Archive:
    1. Publish Madisons-FEP.sar
  6. Enable the Solr foundation
    1. cd 'WCDE_installdir'\bin
    2. enableFeature.bat -DfeatureName=foundation [DsolrHome=C:\IBM\WCDE_ENT70\search\solr\home]
  7. Setup the Search Index:
    1. cd 'WCDE_installdir'\components\foundation\subcomponents\search\bin
    2. setupSearchIndex.bat -masterCatalogId 10001 -instance WCDE_ENT70 -dbuser db2admin -dbuserpwd db2admin [-solrhome C:\IBM\WCDE_ENT70\search\solr\home]
  8. Preprocess the Search index:
    1. cd WCDE_installdir\bin
    2. di-preprocess.bat C:\IBM\WCDE_ENT70\search\pre-processConfig\MC_10001\DB2\ [-fullbuild true] [-localename en_US]
  9. Test the current Solr index to ensure that a request can be recognized:
    1. http://host_name/solr/Default/select?q=*%3A*
  10. Build out the index (first make sure to start the test server in toolkit):
    1. cd 'WCDE_installdir'\bin
    2. di-buildindex.bat -masterCatalogId 10001 
  11. Test the indexed Solr index to ensure results are returning:
    1. http://localhost/solr/MC_10001_CatalogEntry_en_US/select?q=catentry_id:10251
  12. Enable the store to display the search breadcrumb and faceting filters:
    1. Open Management Center
    2. Click "Management Center Tools" menu and select "Store Management"
    3. Select the "Madisons" store and click the "Catalog" tab
    4. check "Search-based navigation"
    5. click Save
  13. Test live on the starter store:
    1. http://localhost/webapp/wcs/stores/servlet/Madisons/index.jsp
    2. http://localhost/webapp/wcs/stores/servlet/Elite/index.jsp


Saturday, February 25, 2012

Using Contracts for eCommerce in Websphere Commerce

As per the request from one of the comments on the blog, this is a high level view on contracts using Websphere commerce and by no means a complete coverage of contracts as it's such a vast topic.

Contracts: This is one of the most interesting concepts when dealing with both B2B and B2C eCommerce systems. Every store published out of the box is associated with a default contract. Lot of clients use contracts on B2C site, if the requirements for a B2C site include using contracts or employment purchase programs using contracts. It is a best practice to use EliteStoreFront or if you are still using V 6.0 (Advance B2B).

Why are contracts required? These are good reasons.
1. If you want to use the same offer price lists as your regular customers but provide a percentage or fixed amount off of the same prices to a subset of customers such as Employees or Students.
2. If you want include or exclude a set of categories or products to a different set of customers.
3. In the case of B2B systems, contracts allow creating separate pricing tiers for different set of customers.
4. Contracts can extend existing contracts or new contracts to provide multi-tiered pricing.
5. You can also have separate shipping prices per contract.
6. Payment methods can also be setup per contract.

How do you create contracts? Essentials for creating a contract.
Organization->Account->Contract

1. Create organization--using orgadmin console.
2. Create Account --(Accelerator, sales-> accounts)
3. Create Contract --(Accelerator, sales->accounts->new contract)

Important concepts related to contracts:
Catalog Filter : Tooling to provide pricing and catalog filters.
Pricing: (Pricing percentage off/fixed amount off) can be used to increment or decrement the default offer price for a subset of customers.
This is provided in Accelerator and Management center. The one is management center is not restricted to a store and can be shared across stores and contracts.
Include/Exclude Catalog: Allows to include exclude catalog specific set of catalog entries by specifying which categories,catalog entries, attribute dictionary attributes.


Friday, February 17, 2012

Solution Faceted Search

Search is an integral part of every company with online presence and corpus data both structured and unstructured. The goal is always to navigate customer find the product/content in a fast efficient way to purchase or view.

What is a facet: As per the Google definition, it's a particular feature or an aspect of something. For those of us from SQL background think of facets as groupBy functionality. As we all understand to perform a groupBy is a complex performance operation from code. In world of searchandize, each facet is modeled as a Dimensions and Dimension Values using a Search engine(It could be any engine Endeca or SOLR or something else)

E.g. Dimensions: Color, Size, Price, Men's, Shirts
Dimensions Values: Red,Blue,Black, White
Ranges: In case where you have e.g. Authors on books or Price where we have a lot of values, it is a good idea to go with a range.

Faceted Search process:
Most of the Search engines have an ETL process to merge\Join data from various sources and massage\clean data, if you are data is not good (e.g. If a company sells USB drive, GB would be a facet but if the value is not clean as an G B or GB. or G.B for GB, it would translate to wrong facets in the front end). hence it is important to have good data.
Once you have get past cleaning data, the process is to build Attribute dictionary and map it to the data and build indices off it.
From the front end, the Navigation is driven by state N=0 that would provide all data from Search and as you drill to specific data, you change the Navigation state. Most of the times, the navigation state is encrypted for front end customers to make the URL smaller. There are other parameters such as Navigation filters.




Monday, February 13, 2012

Endeca | Programmer primer | Objects Query\Results\Connection

A lot of customers use Endeca or SOLR as their search engine in WCS world and this primer series on Endeca is to provide some basics:

Information Transformation Layer: Transforms the source data into indices for MDEX Engine. The transformation process does not change the original data but only it's representation within your Endeca Impl.

Query (Object ENEQuery or sub class UrlENEQuery): Endeca follows one query one page approach. The MDEX is designed to return all of the information required to build an entire page.The MDEX engine is specifically designed this way to reduce the number of queries. There are four kinds of queries:

  • Navigation (N) --setNavDescriptors(),
  • Endeca Record (R)- setERecSpec(), 
  • Dimension Search(D) --setDimSearchTerms(),
  • Aggregate Endeca Record (A, An, Au)--setAggrERecSpec.

e.g. ENEQuery nequery = new UrlENEQuery(request.getQueryString(), “UTF-8”);
     ENEQuery nequery = new ENEQuery();
    DimValIdList descriptors = new DimValIdList("123");
   nequery.setNavDescriptors(descriptors);


Query Search results as EnQueryResults object: For any given type of query such as navigation, record search, dimension search, .etc. The top level object returned is EnQueryResults and that could encapsulate other objects.

Endeca Record Object as ERec Object: It contains all the information regarding the Endeca record including Endeca Properties (name\value pairs) and Information about record's location within the dimension hierarchy.

Connectivity object HttpENEConnection: enables connections with MDEX engine.
     e.g. ENEConnection nec = new HttpENEConnection(eneHost, enePort);


Query Exceptions : There are multiple exceptions ENException, ENEConnectionException objects.


Sample code for retrieval of query results:

//Create an ENEConnection
ENEConnection nec = new HttpENEConnection(eneHost, enePort);
//Create a query from the browser request query string
ENEQuery nequery = new UrlENEQuery(request.getQueryString(),
“UTF-8”);
//Execute the MDEX Engine query
ENEQueryResults results = nec.query(nequery);

Wednesday, February 8, 2012

Endeca overview ---High level Architecture

Endeca is a very widely used search engine for Web and E-commerce applications and it is very popular with lot of WCS clients. High level architecture components of include:

MDEX Engine:  The MDEX engine is the indexing and querying engine that provides the core functionality for all Endeca. MDEX Engine is stateless and hence requires the query to have all the parameters to be provided each time.


Endeca Workbench: The Endeca workbench is a web based tools that allows business users to configure and administer an Endeca application for Merchandising, content spotlighting, search configuration and usage reporting. The main functions are as follows
Rule Manager: Add\Edit\Activate\Deactivate rules and preview using the preview application.
KeyWord Redirects: Create and manage keyword redirects.
There are multiple functions such as Resources Lock Page, Instance Configuration Page, Dimension Order, Stop Words Page, Phrases Page, Thesaurus, Daily Reports,User Management  and EAC settings, EAC Admin Console.

Endeca Platform Services:  This consists of mainly Endeca Information Transformation Layer and Endeca application controller (EAC).
EAC consists of EAC central server and agents. EAC centeral server manages all agents.
Data Foundry: consists of forget program and related components.
Presentation and logging API's: API's to MDEX Engine and Log Server.
Logging and Reporting system.
Reference Implementation: Sample developer studio project source data for multiple platforms.
emgr_update: Utility to update instance configuration to Endeca workbench.
Endeca Control System: The Job control Daemon (JCD) and the control Interpreter.
     Forging \ETL Server: Process of gather data from various sources and building the indices is Forging.
     Pipelines: Data handling components that transforms Raw data to Indices and Deploy them


Endeca Presentation API: provides interfaces to log server and MDEX Server. The API is available in 2 flavors for Unix and Windows.


Friday, February 3, 2012

Securing webserver 404/403 requests from fingerprinting

As a part of improving the security on a eCommerce site, it is always a good idea to turn off fingerprinting.

How to reproduce this: Go to any static asset on the site and change it to wrong file name and you will see a 404 or even 403 errors. It would print all the information regarding the web server, which could be potentially exploited.

IBM_HTTP_Server/X.X.X.X-PMX4623 Apache/2.1.97 (Unix) Server at host.com Port 80

In order to protect the site for security reasons, these values need to be changed in httpd.conf to make the environment more secure. Make this change and restart webserver for the change to reflect.

Test Environment:

ServerTokens Full

Production Env:
           
ServerTokens Prod


Test Environment:

ServerSignature On

Prod Environment:

ServerSignature Off


Thursday, January 26, 2012

Schema.org Microdata for eCommerce sites


A Lot of studies online talk about an increase in 20-30 percent organic traffic using schema.org.
Schema.org provides a collection of shared vocabularies webmasters can use to mark up their pages in ways that can be understood by the major search engines: Google, Microsoft, and Yahoo!

Some Relevant Tags for E commerce tags:
http://schema.org/Product

e.g.:
<div itemscope itemtype="http://schema.org/Product">
  <span itemprop="name">zinog 55 inch TV</span>
  <img src="zinog-tv-55in.jpg" alt="">
  <div itemprop="aggregateRating"
    itemscope itemtype="http://schema.org/AggregateRating">
   Rated <span itemprop="ratingValue">3</span>/5
   based on <span itemprop="reviewCount">11</span> customer reviews
  </div>

http://schema.org/Offer
e.g.
 <div itemprop="offers" itemscope itemtype="http://schema.org/Offer">
    <span itemprop="price">$600.00</span>
    <link itemprop="availability" href=" http://schema.org/OutOfStock ">Out of Stock
 </div>

http://schema.org/Review
e.g.
<div itemprop="reviews" itemscope itemtype="http://schema.org/Review">
    <span itemprop="name">Yo Camera</span> -
    by <span itemprop="author">Yo Boss</span>,
    <time itemprop="publishDate" datetime="2012-01-15">Jan 15, 2012</time>
    <div itemprop="reviewRating" itemscope itemtype="http://schema.org/Rating">
      <meta itemprop="worstRating" content="1">
      <span itemprop="ratingValue">3</span>/
      <span itemprop="bestRating">5</span>stars
    </div>
    <span itemprop="description">The Yo Camera is a revolution.</span>
  </div>


This is interesting for SEO, just thought of adding it here, How Google perceives web pages.
http://www.google.com/webmasters/tools/richsnippets

Ref:
http://schema.org/


Exception| findSEOPageDefintionByPageNameAndStoreID

If you have just created your V7 instance and running into this problem, the problem could be probably hitting the wrong storeId from your browser. The tendency is to look at Stores\WEB-INF\Conf and check the new configuration files for SEO and also blame it on Feature Pack upgrade :) at least I thought so.
 I ran into this problem when I was trying to hit the URL below, the store ID is valid but it was for EliteStorefrontAssetStore
http://localhost/webapp/wcs/stores/servlet/TopCategories_10701_10001


 /GenericJSPPageError.jsp - No expression builder has been configured with the name: "findSEOPageDefintionByPageNameAndStoreID".
                                 javax.servlet.jsp.JspException: No expression builder has been configured with the name: "findSEOPageDefintionByPageNameAndStoreID".
at com.ibm.commerce.foundation.internal.client.taglib.GetDataHandler.execute(GetDataHandler.java:722)
at com.ibm.commerce.foundation.internal.client.taglib.GetDataTag.doTag(GetDataTag.java:388)
at com.ibm._jsp._TopCategoriesDisplay._jspService(_TopCategoriesDisplay.java:813)

Correct URLs for both stores for out of the box are below:

Elite-Login

Madisons-Sign In