Tuesday, December 17, 2019

Product Oriented Development Methodolgy

In several Organizations, the focus of project execution has recently changed from more product-specific deliveries replacing project-specific deliveries. This requires a complete organization-wide change to make every internal/external project run on AGILE ways of working.

What this essentially means is that every project needs to be viewed from an MVP approach. i.e. Most valuable product.  This requires


  1. Changes in the organization and team structures

Teams are more defined based on specific goals. Teams are asked to think more in terms of product innovation and deliver specific features to evolve the product. There is no one-time delivery system and the business teams are aligned directly with these teams to incorporate faster feedbacks on the product roadmaps. Each team is attached to a dedicated product owner who helps create the specific goal required by the business teams. 

  1. Changes to the ways of working.

Different stakeholders (Both IT and business) meet in every few weeks and define the product goals. Teams are more cross-functional so that they can be moved around once the specific need of the goal is achieved. From the business perspective, the money allocation to building these products is more incremental and on a quarter by quarter basis rather than estimating long term projects. If an idea is not flying it can be terminated at any point in time. Teams are more strengthened with Agile and DevOps culture. It’s a more flat structure where the individual teams mostly interact with an Agile Coach, PO, and a PMO in the hierarchy. 

  1. Product Goal Definition

The product goal definition is based on the lifecycle of the idea from start to production and the idea is split into the specific weeks road map. The goals are created based on different objectives related to parameters that can be real-time customer behavior or any quantifiable metric. Each team is asked to present the product's progress in incremental demos every second week. Several of the goals are also based on the market competition and the main idea is faster velocity. 

Sunday, December 15, 2019

Simple way to create automatic alerts for your application using Slack API

Simple Springboot application for Slack notification

Slack has become one of the most widely used channel for communication between teams in many organizations. It has an open source API that can be used to notify alerts and events to different channels and stakeholders.

Below is a simple spring boot application that uses JPA repository to fetch database contents and communicates to Slack via JSlack api



The spring boot application is present in the following path on github.

https://github.com/shailendrabhatt/spring-boot-slack-notification

Saturday, December 7, 2019

Quick 5 Point Website Inspection for Peak Holiday Traffic

The holiday season is around the corner and with the code freezes, teams fear to risk any major changes to production to hamper the critical business time of the year.
Below are quick 5 pointer non-code checks that can help prepare and enhance the website holiday traffic and sales.

1) Examine the Infrastructure Sizing

If on-premise servers are running critical applications, recalculate the approximate capacity or how much traffic these applications can hold up.
This is based on different data points and includes the number of requests per second, average application server response times, instances and their details of CPU, cores, threads, sockets, etc. All the required information typically can be gathered from the analytics or monitoring tools.

Calculating the number of Cores?

For doing this all that is needed is to load the CPU information (lscpu) and view the information related to Threads per socket, Cores per socket and several Sockets.

Calculate the maximum load or throughput of the system?

The next step is to calculate the number of average requests to the application servers. This can again be calculated by viewing the data of the monitoring tool. The information needs to be fetched for the most peak traffic or expected peak traffic for an application or website.

Calculate the Average response times?

The next value that needs to be calculated is the average response times from the application server. This information also is available using any monitoring tool or can also be calculated by using the expected average value in seconds.
Now with the required information in place, the number of cores can be calculated using the formulae
Number of cores = Requests per second * average response time in seconds

2) Tweak the Enterprise Search Relevancy

Several Research articles show that Browse and Search pages are the most visited pages as well as the pages where most customers are lost. Several websites spend a lot of money and time on external Search engine optimization and one of the key features that are overlooked is the Relevancy Ranking of products on the website browse and search.
Below are some tips that can help boost the relevant products to the top and bury the least preferred products on the site.

Prioritization of Attributes

Attributes precedence is key for relevancy. There needs to be a clear definition of the data associated with these attributes. Special considerations have to be taken for data like Category Name, Product Name, Descriptions and other keywords, as these attributes form the basis of the Search Engine Index. Custom attributes can be created as required and added to the prioritization list.

Monitoring Search Results

Use analytics to monitor user behavior on the search and browse results. Information like most clicked products and categories, most popular searches for a product and category, error results, zero product search, the search result with auto spell correction, page behavior, etc indicates clearly how relevant results are shown to the user. These data form the basis of boosting the required results and ignoring irrelevant results.

Enterprise Search Engines

The setting of core search functionalities like Synonyms, Thesaurus, Stemming, Spell Corrections, Did you mean? play a vital role in displaying the relevant results. For sites having multiple locales update custom stemming files with the top key search words. Also, it’s very essential to keep building the dictionary as and when new data sets are added.

3) Keep a Close Vigil on the Browser Load times

Page load times matter a lot during the peak traffic time. While application health is one aspect of evaluation, the end customer load times also need to be closely monitored.
Below are a few of the areas that need to be evaluated so that the browser load times are not hampered.

Evaluate the Caching TTL

Retrieving static content over the public network is not cheap. Larger the file size larger is the bandwidth and higher cost and lower response times. Caching of static content in the browser plays a critical role in reducing those server calls resulting in faster performance and quick page loads.
Ensure HTTP cache is enabled which enables servers to direct the browser cache to serve the content for a longer duration.
If CDN is used, reevaluate the TTL for those static contents especially the images.

Avoid those Content Publishes

Every content push to the live site requires clearing of caches. Try avoiding the risks of these content pushes during the peak times and if required ensure that only the specific content cache is cleared.

Avoid those External Scripts Calls

All the third-party integrations that the website refers to especially from the browsers have to be looked at. During peak traffic hours every external script is extremely vulnerable and can bring down the browser load times and in some unfortunate cases, the website as well.

4) Tune Database Performance

Evaluate the Relational database performance to prevent those long-running queries. Below are a few of the basic tips that can help optimize the database in a quick time.

Evaluate the Database Statistics

One of the critical aspects of tuning the database is collecting and understanding the statistics. While the monitoring tools give certain information on the most time-consuming queries, the statistics collected from the database server can give detailed information about specific tables, the index distribution, details related to database sizing, etc.

Optimize Indexes

It is vital to have the right insight of indexes on tables. This is the most common issue found when dealing with RDBMS. It’s very essential to understand the fine gap between no indexes and too many indexes, Indexing strategies for inserting, reading and updating data.
Most of the modern databases give information regarding columns that are required to be indexed, their priority order, missing or incorrect indexes, etc and also provide required suggestions on improving them.

5) Network Optimization

Retrieving website over enterprise network is underestimated. Responses within the internal organization network require many round trips between the enterprise internet gateway, firewalls, load balancers, security gateways and those web and application servers placed in different data centers. Below are some suggestions that can help optimize the internal network.
Try to remove those additional firewall rules which can gain few milliseconds for every request.
Try to keep the internal network free from any excessive internal traffic.
Identify those infrastructures with a single point of failure and monitor them closely.
Look out for those Bots and Hostile IP addresses that are clogging the network bandwidth.

Thursday, December 5, 2019

How to develop a quick load test for your micro service application

Ensuring the performance and scalability of the applications is crucial. Load testing is a key practice to identify bottlenecks and optimize your Spring Boot microservices. This article explains the process of developing a quick load test on Spring Boot microservice using Apache JMeter.

Load Test Setup:

Follow the below steps to set up a quick load test:

Step 1: Install JMeter Plugins (Optional)
If you haven't installed the JMeter Plugins Manager, do so by following the instructions on the official website: https://jmeter-plugins.org/wiki/PluginsManager/

Once installed, open JMeter, go to "Options," and select "Plugins Manager" to install the necessary plugins for your load test.

Step 2: Create a New JMeter Test Plan
Open JMeter and create a new Test Plan:

Right-click on the Test Plan node.
Choose "Add" > "Threads (Users)" > "Thread Group."
Set the number of threads (virtual users) and loop count according to your testing requirements.

Step 3: Add HTTP Request
Right-click on the Thread Group.
Choose "Add" > "Sampler" > "HTTP Request."
Configure the HTTP Request sampler with your microservice's endpoint details.

Step 4: Add Listeners for Monitoring
To analyze the test results, add listeners:

Right-click on the Thread Group.
Choose "Add" > "Listener."
Add listeners such as "View Results Tree," "Summary Report," and "Response Times Over Time."

Step 5: Configure Timers (Optional)
To simulate realistic user behavior, add timers:

Right-click on the Thread Group.
Choose "Add" > "Timer."
Configure timers like "Constant Timer" or "Gaussian Random Timer" to introduce delays between requests.

Step 6: Run the Load Test
Save your test plan and run the load test:

Click on the "Run" menu and choose "Start" or use the green "Play" button.
Monitor the results in real-time using the listeners you added. Pay attention to response times, throughput, and error rates.

Step 7: Analyze and Optimize
After the test completes, analyze the results to identify performance bottlenecks. Optimize your microservice accordingly and iterate the load testing process.

Conclusion:

Load testing with JMeter is a powerful way to ensure the performance and scalability of any Spring Boot microservices. By following these steps, you can quickly set up a load test, identify potential issues, and fine-tune your microservices for optimal performance in a production environment. Remember to regularly conduct load tests to catch performance issues early in the development lifecycle.

Building Microservices by decreasing Entropy and increasing Negentropy - Series Part 5

Microservice’s journey is all about gradually overhaul, every time you make a change you need to keep the system in a better state or the ...