Friday, August 15, 2014

Bootstrapping Solutions to Market

Bootstrapping Solutions to Market
This blog describes how to augment the WebSphere® Commerce store archive (SAR) to match the relevant business requirements for your solution. For example, the SAR will provide you with some sample data which establishes the store finder feature in the online store.  If you switch the sample data with your actual brick-and-mortar stores, then you could deploy the SAR quickly and demonstrate the store finder feature to your stakeholders with greater impact.

This blog will outline the SAR and give you the insight you need to be able to transition the out-of-the-box software into a bootstrap ready software solution to match your business needs.
Section 1. Overview
In this blog, you will learn how to set up a WebSphere Commerce store archive (SAR) to deploy your solution. These starter stores (SAR) are intended mostly for jump starting your solution outline, however, we can take it a step further and incorporate engineering processes to create bootstraps.  To get more information on bootstrapping within a solution, you could benefit from reviewing the following patent that I have submitted through the US patent office which outlines the benefit of bootstrapping in your solution delivery:
In a more involved and sometimes evolved e-commerce application, the store front assets may be totally different from what is provided in a given WebSphere Commerce Starter Store. 
As the development assets evolve through the iterative process, you need to deploy those assets. When delivering a WebSphere Commerce application using an iterative approach, you need a low-cost and convenient mechanism to deploy these assets on an ongoing regular basis. The benefit of this entire bootstrapping exercise is to impose your own branding and ideas directly into the product itself.  Thus, as you evolve in your understanding of the product, the product appreciates this by evolving through the bootstrap mechanism described in this blog.
Internal reusable assets inside a store archive
When deploying a WebSphere Commerce application, there are many assets to consider. The SAR has a mechanism to deploy those as assets as listed below:
Resolving and massloading data You can break up data into several categories: core data, configuration data, managed data, and operational data. Core and configuration data are needed to keep each iteration of the developed assets workable in a store flow. Managed and operational data enable ongoing unit and regression testing of the WebSphere Commerce application.
Resolving and loading Access Control To implement access control in a store at earlier stages of the project, minimize the policy contentions that may emerge later during more time constrained phases of a project. The store archive conveniently resolves and loads access control for each development iteration. 
Publishing business accounts and contracts Accounts and contracts have substantial subsystem touch points that may require complex loading dependencies. To place the account and contract requirements in a store archive, minimize the required deployment skills and resources.
Placement of JSP JSP files, along with their associated Javascript, CSS, and images, have a different placement on a development server as opposed to a server. To enable a developer to quickly share these assets, a store archive can conveniently handle the placement of these assets without the developer's responsibility to define through an external build and deployment tool.
Placement of store property files Property files have a different placement on a development server as opposed to a server. To enable a developer to quickly share these assets, a store archive can conveniently handle the placement of these assets without the developer's responsibility to define through an external build and deployment tool.
Extended sites deployment using component store archives
As this blog focuses on maintaining the asset store archive, WebSphere Commerce provides a store archive specifically for deploying assets for that
Section 2. Setting up the environment
Deploying a Starter Store
Assuming you did not already deploy an IBM WCS starter store in your environment, there is a great resource you can follow in the IBM Information Center where you can search for “deploying a starter store”.   Once you deploy the starter store, you are able to redeploy that same store continually over and over to refresh the bootstrap.  As you change the Store Archive Resource, then your bootstrap evolves, thus refreshing your environment with the bootstrap starter store plus your changes.

Section 3. Studying the store archive
Once you have successfully published a starter store in your development environment, you are ready to start developing the store to fit your business requirements. If you start changing JSP files, coding some new URLs to point to those JSP files, pulling data using data beans, or even simply demonstrate the out-of-the-box features, there is a way to bootstrap the experience.
Locating the SAR
You will need to cross-reference the actual SAR file to which you published your store.
You can track this down by opening the following file: <Commerce Install Directory>/xml/tools/devtools/publish/StorePublishWizardArchiveList.xml, where the view selected when you selected extended sites was the value listed in the Store Publish section of the Administration Console.
Using the selected value you selected in the Store Publish as your reference, open the SAR registry file and cross-reference the actual SAR file that was published, <Commerce Install Directory>xml/tools/devtools/SARRegistry.xml, where the <view name="XXX"/> matches the value to which you selected in your store publish.  There may be several entries for XXX. Specifically, you are looking for the fileName value to match the publish reference, which let's say is the fileName="XXX.sar".
From this element reference, there is also an associated attribute relativePath. In this case, it is relativePath="YYY". The relativePath value of YYY means that you can find the path for fileName in <Commerce Install Directory>/starterstores/YYY/XXX.sar.  This is the SAR file you will take apart and study.
Doing it the Chrysler way
Just as Chrysler took apart the Locomobile in his own home and studied the parts, you will take apart your starter store SAR and do the same. The remaining steps will take that SAR apart into its pieces to take a deeper look.


  1. Copy the SAR found in the starterstores directory and place it into a temporary directory in your own development workstation.
  2. Extract the contents of the SAR file into a subdirectory. When the extended sites XXX.sar is extracted, the following directories will be available:

    YYY/  
    Directory to hold store front assets, such as JSP files.  tools/Assets in this directory will enable the Store preview functions.

    SAR-INF/  
    The publish parameters used by the Publish wizard are defined by the store-refs.xml file in each store archive. WebSphere Commerce unpacks the assets from the store archive to the WebSphere Commerce Server. Unpacking the assets is controlled by the unpack.xml file, located in the SAR-INF directory in the store archive.

    META-INF/  
    This is available when adding a new JAR file into the classpath.

    WEB-INF/  
    This directory contains all things related to the application that are not in the document root of the application. The WEB-INF node is not part of the public document tree of the application. No file contained in the WEB-INF directory may be served directly to a client by the container. However, the contents of the WEB-INF directory are visible to the servlet code using the getResource and getResource-AsStream method calls on the ServletContext, and may be exposed using the RequestDispatcher calls. Hence, if the Application Developer needs access, from the servlet code, to application specific configuration information that he does not wish to be exposed directly to the Web client, he may place it under this directory. Since requests are matched to resource mappings in a case-sensitive manner, client requests for /WEB-INF/foo, /WEb-iNf/foo, for example, should not result in contents of the Web application located under /WEB-INF being returned, nor any form of the directory listing.
  3. Establish what parts of this SAR you will customize:

    YYY/   
    All of the developers' changes to the JSP and store front files will be updated in this directory.

    tools/  
    There will be no changes in this directory for this blog.


    SAR-INF/  
    There will be some special parameters added to the publish steps to be updated in this directory. Specifically, the file store-refs.xml will be updated to reflect the build.

    META-INF/  
    There will be no changes in this directory for this blog.

    WEB-INF/  
    This is where the property file and data changes will go.

    Property Files subdirectory  
    WEB-INF/classes/YYY/

    Data subdirectory  
    WEB-INF/stores/YYY/data/
  4. How the store archive unpacks these files at Publish time
    After you have clicked Finish in the Publish wizard, or run the publishstore utility, WebSphere Commerce unpacks the assets from the store archive to the WebSphere Commerce Server. Unpacking the assets is controlled by the
    unpack.xml file, located in the SAR-INF directory in the store archive.

    The
    unpack.xml file determines which files to unpack (using the include and exclude elements), to which directories the files will be unpacked (using the unpack dest entity), and also renames directories (using the rename-store-dir entity).


    By default, unpack unpacks all of the files of the store archive that it is located in. However, unpack can also unpack just certain files within the store archive, if specified. Also by default, unpack unpacks the files to the path obtained by combining the StoreDocRoot and StoreWebPath paths from the DevTools element in the instance XML. This path points to the document root of the Stores Web module. However, if specified, as in the preceding example,
    <unpack dest="${wc:ToolsStoresPropertiesPath}">, unpack will unpack the files in another location. Note unpack accepts variables. In this case the "${wc:ToolsStoresPropertiesPath}" variable is an attribute of the devtools element in the instance.xml.
  5. Publish the wizard publish parameters The publish parameters used by the Publish wizard are defined by the store-refs.xml file in each store archive.
    Note
    : Publishing parameters are only available through the Publish wizard. If you publish a store archive through the command line, you cannot specify parameter values. The default values contained in the store archive will be used.
    Look at the following example of a
    store-refs.xml file from the YYY.sar file.

    ref id  
    This is used as the key in the properties file specified by the store-refs resource-bundle attribute. It is used to obtain the translatable parameter name and the description that displays in the publish parameters page.

    entity  
    This is the name of the ENTITY in the target-dtd that is edited by this parameter.

    input type  
    This controls how the parameter is displayed on screen. If the input type is text, the parameter is displayed in an editable field. If the input type is member, all of the existing organizations display in a drop-down list. You cannot edit read-only parameters.

    target-dtd  
    This defines the target-dtd file. The values that a user enters for these parameters are stored in the file identified in the target dtd file. This file is also part of the store archive and is unpacked with the store data assets. The entity value corresponding to each parameter is updated in the unpacked file. The DTD inside the store archive is not updated. The values for the parameters are stored in this file (in this case ForeignKeys.dtd) until the publish is instantiated.

    resource-bundle="/SAR-INF/properties/publishNLS"  
    Finally, if a store is published in several languages, as the starter stores are, the publish parameters and their accompanying descriptions are found in locale specific files. The field label and description for each publish parameter are located in the properties file defined in the resource-bundle attribute of the store-refs.xml.
    During publish, the process looks for the specific locale for the language used in the Publish wizard. The
    stores-ref.xml file also defines these files.

    deploy-descriptor="WEB-INF/stores/XXX/data/ibm-wc-load.xml  
    The deploy descriptor specifies the location of the file (ibm-wc-load.xml) that controls the publish data portion of the publishing process.
    You should examine how the data is handled by the store archive. The data loading segment of the store archive can handle loading core data, configuration data, managed data, and operational data. The store archive can also handle loading special complicated data, such as access control and contracts. The next section will focus on the data loading segment of the WebSphere Commerce store archive.
Section 4. Data in the store archive
The WebSphere Commerce Store Archive has a very sophisticated manner to handle data. The data that is to be loaded into WebSphere Commerce is represented as XML files. You can have multiple XML files, each can handle a specific set of tables, or even a subsystem. A master file dictates which files are loaded in which sequence. That way, you can set data with special unresolved identifiers so that the SAR can resolve and manage the massload responsibilities at runtime. With this design, you can package data into the WebSphere Commerce store archive and load it on any database on any node without having to hard-code database specific keys.
The unresolved way
When data is represented with unresolved identifiers, this allows the publishing process to massload the data without having to error with primary key conflicts. To learn more about using internal alias to resolve identifiers, see Creating an XML file to use internal-alias ID resolution.
External dependencies
The Publish wizard publish parameters from the store-refs.xml file specifies a particular file, the value of target-dtd, to designate external dependencies. As the path is specified from the target-dtd parameter, it is found directly within the data subdirectory: WEB-INF/stores/YYY/data/ForeignKeys.dtd. You will be adding some new parameters to this file to make some special changesto fulfill this blog requirement.
The deploy descriptor
The Publish wizard publish parameters from the store-refs.xml file specifies a particular file, the value of deploy-descriptor, as the deployment descriptor. As the path is specified from the deploy-descriptor parameter, it is found directly within the data subdirectory: WEB-INF/stores/XXX/data/ibm-wc-load.xml. You will be adding new accounts and contracts to be deployed in this blog.
Of specific interest to this 
blog from the deployment descriptor are the master and resolved.master assets. These are denoted as follows:
XML example 3. Deployment descriptor to loading data assets
<asset id="master" location="store-data-assets.xml"/> <asset id="resolved.master" location="store-data-assets.resolved.xml"/>

The
master asset is the file that controls which XML files that are to be massloaded during the store archive publish process. When considering unresolved identifiers in the data XML files, those unresolved entities that are designated as the primary key should be sequenced before other XML data files (which hold those same unresolved identifiers as foreign keys).
The
resolved.master asset is the output of the resolved assets that are listed in the master asset file. This resolved file is important during the troubleshooting phase of the publish process when there are resolving or massload errors.
Section 5. A development scenario
For this blog to represent a real-world scenario, there are many WebSphere Commerce sub-components that have to be taken into consideration. Therefore, this blog will include the following development scenario:

  1. Creating a catalog data to reflect the entitlement.
  2. Creating new views, commands, and JSP files to display the data.
  3. Mapping the new views and commands into the Access Control.
  4. Finally, putting all of those assets into the WebSphere Commerce store archive and publishing them to a new environment.
To start with these development steps, you should review Modify asset store archives flow (Enterprise). As you can see from this reference, repackaging the store archive will be an iterative process as you go through the decision points of what assets need to be updated during development.
Section 6. Creating data in the store archive
Creating data means updating the file from the master asset, store-data-asset.xml, in the deployment descriptor.
Each starter store archive contains a store-data-asset.xml file. The store-data-asset.xml file includes placeholders for all of the data asset files in the store archive that will be included during the publish.
The following is a portion of the store-data-asset.xml file for the ConsumerDirect.sar, illustrating the placeholders:
With that change, there needs to be an associated change in the store-data-assets.dtd file to map these entity references.
During the publish, all the data assets identified with placeholders in the store-data-asset.xml file are consolidated into one large document. The ID Resolver resolves the IDs found in the document. After the IDs are resolved, ID Resolver creates the following file, store-data-asset.resolved.xml, which contains unique IDs. If an error occurs during the ID resolving process, the Loader package adds an entry to the messages.txt file.
You will create these XML files and add the corresponding data to them.

The next few sections will outline changes that can be made in the SAR.  Some data fits well to bootstrap while other types of data does not fit.  For example, the catalog data is not the best candidate to bootstrap into the SAR.
The catalog data is usually externalized into a separate standalone SAR file.  It is recommended to use BODL to load the catalog data due to the large amounts of information.  With that, you most likely would not deploy catalog data with your starter store.

Section 8. Store Locator Data

When deploying a SAR to demonstrate the IBM WCS functionality to your project stakeholders, it would be nice if that SAR contained the stores relative to your project.   You can actually provide this by bootstrapping those project stores into the IBM WCS product SAR.

The following steps is an general outline you can use to follow to bootstrap the stores into the SAR:

  1. Generate Store Locator data file.
    1. Pull store locations from HD.ca live
    2. Pull coordinates of store locations from http://itouchmap.com/latlong.html
    3. Map the data into the store locator tables
      1. GEONODE contains the basic city information.
      2. GEOTREE contains the associate reference to GEONODE
      3. STLOC contains the store information including the latitude and longitude along with the GEOTREE stated above
      4. STLOCDS language support for the store information
      5. STLOCATTR contains the store hours
  2. Extract the following SAR:C:\IBM\WCDE_ENT70\starterstores\YYY\XXX.sar
  3. Review the following files to ensure there is a reference to "store_locator.xml" and if there is not one that exists, created it and append to the following:
    1. store-data-assets.dtd
    2. store-data-assets.xmlthese files are updated in the following directory:C:\install dir\starterstores\YYY\XXX\WEB-INF\stores\StoreAssetsDir\data\store-data-assets.xml
Section 9. Views and commands
The command, view, and URL registries are part of the WebSphere Commerce command framework. The command framework determines how a command executes, and then returns a response based on the view returned by the executed command. The command execution and response are store dependent, which means that you can implement the same command differently for each store, as well as return different responses for each store.
For purposes of this blog, you will simply reuse an existing view and rename it. Thus, you save those additions in the store archive.
Creating and deploying a view
After a command is executed, in most cases, the requester of the command requires a response to be returned. When determining the response, the command framework considers the following factors:


  • The view found in the response properties after the command is executed.
  • The store on whose behalf the command was executed.
  • The device format of the request when the request was made.

Every view that returns a response must be defined in the view registry, either per store, or by default, by site. Each store will normally define the view for each possible device format of the incoming request. However, if a view is not defined by a store, the default view for the site will be used. The adapter handling the request will decide which device format and the default device format to use when determining which view to call. There is no one generic device format, so depending on the different types of requests that can be accepted by WebSphere Commerce, there may be a view defined for each device format.
For this blog, you will be creating a new JSP and enabling it as a view in the store archive. To accomplish this task, you will do the following steps:


  1. Duplicate the JSP for the TopCategoriesDisplay page.
  2. Create the view data in the command XML file of the store archive.
  3. Create the Access Control for that view in the Access Control XML file of the store archive.

Creating the JSP

When you first deployed ExampleStorefrontAssetStore during the initial publishing stages to set up this environment, there was a set of JSP that were deployed from the SAR file. Go to the extracted directories of the YYY.sar, and locate XXX/.

Within that directory, navigate to the subdirectory ShoppingArea/CatalogSection/CategorySubsection/. Within that directory, there should be a JSP file called TopCategoriesDisplay.jsp and CachedTopCategoriesDisplay.jsp. Make a copy of these two files and rename them to ExampleCategoryDisplay.jsp, and leave CachedTopCategoriesDisplay.jsp to still be referenced by the new JSP file.

Now that you created the new JSP file, add the view ExampleCategoryDisplayView.

The following is the data file that was previously set in store-data-assets.xml as the ExampleStorefrontAssetStore_command.xml entity.

View data

Create the ExampleStorefrontAssetStore_command.xml file and create the corresponding data.

Now that the view is set to be registered in WebSphere Commerce, you will need an associated access control definition.

Access control data

Create the ExampleStorefrontAssetStore_accesscontrol.xml file and create the corresponding data.

acaction_id="@ExampleCategoryDisplayView"/> <acactdesc acaction_id="@ExampleCategoryDisplayView" displayname="Example Category Display View" description="Example Category Display View" language_id="&en_US;"/>

Notice from this Access Control definition that we are reusing the @xxxAssetStoreRegisteredCustomerViews unresolved identifier that was declared in the existing store archive assignment in the accesscontrol.xml file.

Now that the JSP and view are ready to be deployed into WebSphere Commerce, you will now perform your first iterative build.
Section 10. The first build

Assuming that the extended sites has been deployed and a store has been opened, you can deploy the first development build. When the previous sections of this blog have been successfully completed, there should be an update to the extracted version of XXX.sar. From the extracted directories, you will need to zip those directories back up into an updated version of XXX.sar.

For this example:

Command example. Extracting a WebSphere Commerce store archive
cd <DevelopmentDirectory>\testWithin the directory structure that you updated the SAR contents, package that back up into a sar file which reflects the same structure as when it was initially extracted.  You can achieve this either using JAVA (jar -cvf XXX.sar <DevelopmentDirectory>\test) or 7zip or WinZip.

Now that your WebSphere Commerce store archive is ready for publishing, you will need to use the publish tool. For different options, see Publishing a Store Archive in the online IBM WCS Information Center. The following is an example of using the Publish Wizard to successfully publish this updated SAR file:


  1. Log into the WebSphere Commerce Administration Console tool.
  2. Select the menu titled Store Archives, then click the sub menu Publish, the Publish page will show the Store Archives panel.
  3. Select options for the View dropdown (or select the default Store Archives view, and then reselect the option Store Archives view.
  4. Check the SAR file XXX.sar.
  5. Click the Next button on the top right of the panel. The Parameters panel appears.
  6. Ensure the parameters match the settings you adjusted in your changes
  7. Click the Next button on the top right of the panel, and the Summary panel appears.
  8. Accept the defaults and click the Finish button on the top right of the panel.
  9. Click the OK button for the Web page dialog.
  10. Wait for the publish to complete and click Refresh to acknowledge the Publish Status as complete.
After you have completed publishing this updated WebSphere Commerce store archive, you can begin testing your changes. When you log into the store this time, you can logon as usual, then once logged into the store, you can enter in the URL which corresponds to your new view name: ExampleCategoryDisplayView (for example, http://localhost/webapp/wcs/stores/servlet/ExampleCategoryDisplayView?langId=-1&storeId=10151&catalogId=10001; where langId, storeId, and catalogId values may be different values for your store).
Section 11. Subsequent builds
For subsequent builds, follow the steps outlined in this blog. Upon each new version of the WebSphere Commerce store archive that you would append to, simply update the directory and file, <WC_installdir>\starterstores\XXX\YYY.sar, over and over, thus evolving your application with a packaged and controlled deployment mechanism.
Keeping in mind that you have been updating the Access Control policies for all your new commands and views. It would be wise to continually refresh your environment upon each deployment. This means reverting your database back to the WebSphere Commerce bootstrap before you publish your first developed SAR. A total refresh is really not a problem for your development team anymore. It takes as long as a SAR publish would take to bring your environment back up from scratch!
When refreshing your environment, you ensure that the access control policies imposed by the bootstrap WebSphere Commerce system is in place. Then your latest SAR will add-on the remaining access control delta policies, which apply to your latest user access requirements for commands and views.
So, now that you know you are going to repeat the deployment of your SAR over and over again, then you can setup your Publish wizard publish parameters within the store-refs.xml to be reflected towards parameters you know you are publishing. For example, our initial store-refs.xml from the original SAR had the following parameters:
<ref id="storeDir" entity="STORE_DIR" ><input type="text"/></ref><ref id="storeIdent" entity="STORE_IDENTIFIER" ><input type="text"/></ref><ref id = "parentOrg" entity="ORGANIZATION_DN"><input type="member" /></ref></store-refs>

Since you are most likely going to publish your catalog data and your store data along with your B2BDirectStorefrontAssetStore SAR implementation, you can add the following entries to your store-refs.xml as a way to get your data setup for the XML data that may need to reference it.

The above settings provide your SAR the targetStore identifier, which uses a drop down supplied with some selectable data as input. The selection is already set to some data that was represented in the ForeignKeys.dtd file to populate the identifier, memberDN, and directory. The hubStoreContract identifier pulls a parameter from the ForeignKeys.dtd file and sets the entity reference: hub_created_store_contract_1. The identifiers are used as keys to pull localized verbage from the publishNLS.properties file in the properties directory of SAR-INF. The entity reference carries a value to be used in the mapping of the ForeignKeys.dtd, which is used to populate data during the SAR publishing. To review this, open the target-dtd="WEB-INF/stores/XXX/data/ForeignKeys.dtd"file reference noted from this store-refs.xml file.

Within the ForeignKeys.dtd" file, you can add a new entity reference and even include hard-coded values as necessary.

With the publish parameters updated and with some unique ways of complimenting your development builds, you can see how powerful it is to use the SAR as your deployment mechanism.

Section 12. Conclusion

Leveraging the store archive as your development, build, and deployment mechanism saves a lot of resources required to build ANT scripts to do what your SAR would already provide for you. However, IBM WCS does provide a Build-and-Deploy tool which already provides you with all the ANT logic that you require.  So, then why even bother with the SAR augmentation?  Twofold: the SAR can be passed to your team or even back to the original product team for support. This compliments Agile processes to increase response time confirm what has been engineered in the solution directly matches the bootstraps provided through the SAR.  Also, the benefit is that you directly align your solution to the bootstrapping of the product.  

In the future, I would challenge that IBM WCS merge their SAR (starter stores) directly into the Build-And-Deploy tool.   That way, when you bootstrap, you are modelling that directly into the build process.  What this means is that when you first install IBM WCS, you actually install a starter store from the build process.  Good idea?  Let IBM know so that you can start boostrapping into the build-and-deploy tool instead of the SAR.