- EngineeringWay

EngineeringWay

Shaping the great minds.

Friday, 5 January 2018

How salesforce handle 1.3 Billion Transactions A day and 24,000 database transactions per second - Technology explained!

1/05/2018 09:53:00 am
Salesforce.com is interested in being more open with the technology communities that we have not previously interacted with. Here’s to the start of “Opening the Kimono” about how we work.
Since 1999, salesforce.com has been singularly focused on building technologies for business that are delivered over the Internet, displacing traditional enterprise software. Our customers pay via monthly subscription to access our services anywhere, anytime through a web browser. We hope this exploration of the core salesforce.com architecture will be the first of many contributions to the community.
Salesforce platform is built on Oracle backend database, not just but a cluster of databases. They have built a layer of abstraction over that and you cannot access the database directly, but use their database queries (soql).
there are about 140 instances/nodes across NA, EMEA and APAC.  All customers in the world are assigned one of these nodes in their geographic region, meaning many customers share one instance.

Stats 

  • 17 North America instances, 4 EMEA instances and 2 APAC instances
  • 20 sandbox instances
  • 1,300,000,000+ daily transactions
  • 24,000 database transactions per second at peak (equivalent to a page view on other sites)
  • 15,000+ hardware systems
  • > 22 PB of raw SAN storage capacity
  • > 5K SAN ports

Software Technologies Employed

  • Linux for development and primary production systems
  • Solaris 10 w/ ZFS
  • Jetty
  • Solr
  • Memcache
  • Apache QPID
  • QFS
  • Puppet, Razor
  • Perl, Python
  • Nagios
  • Perforce, Git, Subversion

Logging In To The Salesforce.Com Service

We maintain a pool of servers to handle login traffic for all instances. A handful of servers from many (but not all) instances accept login requests and redirect the session to the user's home instance. This is what happens when you log in via login.salesforce.com.
Customer traffic starts with our external DNS. Once a lookup has successfully returned the IP address for an instance, standard Internet routing directs it to the appropriate datacenter.
Once the traffic enters our network in that datacenter, it is directed to the load balancer pair on which that IP lives. All of our Internet-facing IPs are VIPs configured on an active/standby pair of load balancers.

Inside The Instance

The load balancer directs the traffic to the application tier of the given instance. At this tier, we service both standard web page traffic as well as our API traffic. API traffic makes up over 60% of the traffic serviced by our application tier overall. Depending on the needs of the customer's request, it will be directed to additional server tiers for various types of backend processing.

Core App

The core app tier contains anywhere from ten to 40 app servers, depending on the instance. Each server runs a single Hotspot JVM configured with as much as a 14 GB heap, depending on the server hardware configuration.
The batch server is responsible for running scheduled, automated processes on the database tier. For example, the Weekly Export process which is used to export customer data in a single archive file format as a form of backup.
Salesforce.com offers a number of services including basic and advanced content management. We have a content search server and a content batch server for managing asynchronous processes on the content application tier. The content batch servers schedule processing of content types, including functions such as rendering previews of certain file types and file type conversion.

Database

The primary data flow occurs between the core app server tier and the database tier. From a software perspective, everything goes through the database so database performance is critical. Each primary instance (e.g. NA, AP or EU instances) uses an 8 node clustered database tier. Customer sandbox (e.g. CS instances) have a 4 node clustered database tier.
Since salesforce.com is such a heavily database-driven system, reducing load on the database is critically important. To reduce load on the database tier, we developed ACS -- API Cursor Server. This was a solution to 2 problems which enabled us to improve our core database performance significantly. First, we used to store cursors in the database but the deletes were impacting performance. Second, after moving to using database tables to hold cursors, the DDL overhead became a negative impact. Thus was born the ACS. ACS is a cursor cache running on a pair of servers, providing a method to offload cursor processing from the database tier.

Search

Our search tier runs on commodity Linux hosts, each of which is augmented with a 640 GB PCI-E flash drive which serves as a caching layer for search requests. These hosts get their data from a shared SAN array via an NFS file system. Search indexes are stored on the flash drive to enable greater performance for search throughput.
Search indexing currently occurs on translation servers which mount LUNs from storage arrays via Fibre Channel SANs. Those LUNs make up a QFS file system which allows single writer but multi-reader access. Like most other critical systems, we run these in active/passive with the passive node doing some low priority search indexing work. It then ships its results to the active partner to write into the QFS file system.
The translation occurs when these same LUNs are mounted read-only from a group of four NFS servers running Solaris 10 on SPARC. These SAN mounted file systems then are shared via NFS to the search tier previously described.

Fileforce

We maintain a tier of servers that provide object storage, similar in concept to Amazon's S3 or OpenStacks' Swift project. This system, Fileforce, was developed internally to reduce the load on our DB tier. Prior to the introduction of Fileforce, all Binary Large Objects (BLOBs) were stored directly in the database. Once Fileforce came online, all BLOBs larger than 32 KB were migrated into it. BLOBs smaller than 32 KB in size continue living in the database. All BLOBs in Fileforce have a reference in the database so in order to restore Fileforce data from backups, we have to start a database instance based on a database backup from the same restore point.
Fileforce includes a bundler function, developed to reduce the disk seek load on the Fileforce servers. If 100+ objects smaller than 32 KB are stored in the database, a process runs on the app servers to bundle those objects into a single file. A reference to the bundled file remains in the database along with a seek offset into the bundle. This is similar to Facebook's Haystack image storage system but built into an object storage system.

Support

Each instance contains various other servers for support roles such as debugging application servers and "Hammer testing" app servers in the app tier, hub servers which monitor each instance for health and monitor servers running Nagios. Outside of the instance itself reside supporting servers like storage management, database management, log aggregation, production access authentication and other functions.

 Salesforce database
I hope this overview of the salesforce.com technology architecture and stack has been interesting and informative.Thanks for reading!

Thursday, 21 December 2017

what's Blockchain technology? bitcoin blockchain ,database and bitcoin wallet explained - Engineeringway

12/21/2017 08:31:00 am
Last year, ICICI Bank announced that it successfully executed transactions in international trade finance and remittances using blockchain technology in partnership with a Dubai based bank Emirates NBD.
In 2008, a cryptographer who goes by the pseudonym Satoshi Nakamoto created a crypto-currency called bitcoin. Bitcoin is digital currency that allows you to perform peer-to-peer transactions without the help of a third party such as banks.
With a blockchain, many people can write entries into a record of information, and a community of users can control how the record of information is amended and updated. Likewise, Wikipedia entries are not the product of a single publisher. No one person controls the information.
Descending to ground level, however, the differences that make blockchain technology unique become more clear. While both run on distributed networks (the internet), Wikipedia is built into the World Wide Web (WWW) using a client-server network model.

  • What is blockchain technology?

A blockchain is an anonymous online ledger that uses data structure to simplify the way we transact. Blockchain allows users to manipulate the ledger in a secure way without the help of a third party.
A bank's ledger is connected to a centralised network. However, a blockchain is anonymous, protecting the identities of the users. This makes blockchain a more secure way to carry out transactions.
The algorithm used in blockchain reduces the dependence on people to verify the transactions. This technology used for recording various transactions has the potential to disrupt the financial system.

  • How it works?

blockchain enables two entities that do not know each other to agree that something is true without the need of a third party. As opposed to writing entries into a single sheet of paper, a blockchain is a distributed database that takes a number of inputs and places them into a block. Each block is then 'chained' to the next block using a cryptographic signature. This allows blockchains to be used as a ledger which is accessible by anyone with permission to do so. If everyone in the process is pre-selected, the ledger is termed 'permissioned'. If the process is open to the whole world, the ledger is called unpermissioned.

bitcoin blockchain ,database and bitcoin wallet explained

Transactions are broadcast, and every node is creating their own updated version of events.
It is this difference that makes blockchain technology so useful – It represents an innovation in information registration and distribution that eliminates the need for a trusted party to facilitate digital relationships.
Yet, blockchain technology, for all its merits, is not a new technology.
Indian IT service providers like Infosys and TCS have been throwing their weight around blockchain technology. Both these companies are using blockchain mechanism to create core banking platforms for banks.

  • Where can it be used?

Use of blockchain technology is not limited to the financial sector. It is being used in many other areas. For example, Honduras government has put all land records on a public ledger - the blockchain. The minute there is a change in ownership, it gets recorded publicly.

  • Is it safe?

The USP of blockchain is that it allows two parties to execute a transaction without any intermediary. Blockchain allows financial institutions to execute and verify transactions discretely without any human intervention.
The electronic ledger of transactions is continuously maintained and verified in 'blocks' of records. With the help of cryptography, the tamper-proof ledger is shared between parties on computer servers.
Experts believe that blockchain architecture can significantly bring down the costs and reduce inefficiencies in the financial sector.

Saturday, 16 December 2017

How to generate links that drive traffic, not just ranking. - searchengineland, SEO

12/16/2017 11:00:00 pm


Links are a crucial element of search engine optimization, and columnist Kevin Rowe believes that long-term SEO success relies on building links that drive real traffic.
What’s so great about referral traffic? Do you really have to ask?! Referral traffic is great because it gets your content in front of new audiences, creating new opportunities for audience engagement and conversions.

Many people see link building as a way to drive rankings. But, when done correctly, it can (and should) also drive traffic.

Driving traffic has a lot of benefits beyond the obvious potential increase in leads and sales. More website traffic can provide valuable analytics data about what users are looking for and what confuses them. It can also help grow engagement and potentially referral links on social media as others begin to share our content.

In this column, I’ll explain how to identify sources of links that drive actual traffic and how to evaluate your progress so that you can focus your efforts where they will have the greatest impact.

Identifying link partners

In order to find good sources for traffic-driving links, there are a few ways you can go: competitor research, rankings and influencers.

First, find the publications driving traffic to your competitors by using tools like SimilarWeb to find their top referral sources. Not only do these tools tell you who is linking to your competitors, but some can also show how much traffic your competitors are getting from those links.

Any site driving traffic/referrals to your competitors should be investigated and evaluated as a potential linking partner. Check each one for quality, verifying that they aren’t content scraper sites and are actually valuable resources for your target audience. If they pass the test, then consider approaching them for a link.

Of course, you shouldn’t just pursue links from sites that are driving traffic to your competitors. Review the top-ranking websites in Google for the terms you want to rank for and see if any of them can serve as good linking partners. For example, many industries have vertical-specific directories that provide both free and sponsored listings.

As always, do your research when approaching sites like this. Do the directories seem spammy, designed only to generate links for SEO purposes? Or are they legitimate sites that consumers actually use, like Yelp, TripAdvisor or Avvo? (Note that links from legitimate sites will often be nofollowed, but they are still valuable because they drive real traffic.)

If you want to do more of the heavy lifting when it comes to content, try approaching major and niche industry outlets that you can contribute content to. In addition to the above sites you found during your research, use a tool like BuzzSumo to find social influencers and reach out to them on their social channels or via email to see if they accept guest posts. These posts need to be highly relevant to the website’s audience, and be careful to follow any editorial guidelines and respect their rules for submitted content.

One last angle to try is to find industry influencers and sponsor or partner with them. Many influencers are willing to enter into partnerships with brands, where they will review or work with a company on content and social media posts to get the brand’s name out to their audience. Cost usually varies with audience size and the scope of the campaign.

Since the aim here is to drive traffic and branding, you shouldn’t run into any issues regarding Google’s linking guidelines. However, it’s important to ensure that all financial relationships are disclosed according to FTC guidelines and that you aren’t attempting to hide or sneak links into any content that you are sending to these outlets for publication.
Evaluating success

Once you’ve approached your chosen link partners and successfully obtained links, it’s time to review your work. After each month, check Google Analytics for referral traffic to see which new sites you’ve worked with are actually bringing you traffic. After three to six months, you’ll have a clear picture of which sites are worth your time and which aren’t. For instance, if Inc.com is bringing you more traffic than three industry sites combined, it might be better to pare down your industry sites to be able to submit more content to Inc.com.

Additionally, you can also see if there is an increase in overall brand search for your name using Google trends or Google Keyword Planner. Often, branding campaigns can result in more direct traffic, as well as organic traffic due to an increase in branded searches. By carefully tracking increases in direct and branded organic referrals, you can see the impact your branding campaigns are having. This can help you see the long-term benefits of your link-building efforts in growing your website traffic.

While tracking the data, be sure to also track your success building relationships with the influencers and websites you’ve singled out as potential link-building partners. This can show your progress to management and help you hone your pitch and messaging style.

links from authoritative domains are still influential in the ranking algorithm and they can still be great for branding. But if your manager is on you to increase referral traffic – not just links – focus on the types of links that actually get clicked.

Wednesday, 25 October 2017

10 Hidden but Powerful Google Tools for Business and Marketing – That You Never Heard Of And You Should Be Using.

10/25/2017 09:46:00 pm
You may have heard of this little thing called Google. You know, where 1.17 billion people go to find stuff on the web?
But Google is more than just a search engine. So much more.
In fact, Google offers a ton of tools in addition to its search engine that can be hugely valuable if you're a marketer.
So I decided to round up some of the most essential Google marketing tools at your disposal so you can be sure your business is taking full advantage of all Google has to offer.

1) Google My Business

Want to get yourself some free advertising on Google? I kid you not -- it's a real thing. 
Over 100 Billion searches are performed on Google every month. So, if your business is not discoverable on Google, you are losing out on a huge business opportunity. Fortunately, Google makes it easy for small business to list their business on Google products such as Maps, Google+ and on search engine.
Google My Business is a free tool that lets you list your local business easily. Its a great way to build your web presence and generate more leads.

2) Think With Google

Speaking of seeking data to help your company evolve, don't miss Think With Google. It's a free marketing resource loaded with consumer trends, marketing insights, case studies, industry research, and creative inspiration.
Think With Google is a nice place where you can get useful articles, various infographics and interviews of industry leaders. This site is updated constantly with loads of useful content that you can use to grow your business.
It also has collection of creative AD campaigns you can draw inspiration from.

3) GoMo

Did you know that 67 percent of people say a mobile-friendly site makes them more likely to buy a product or use a service? (Or that even if these people like your business, 50 percent will use you less often if your website is not mobile friendly!) Don't fall into the latter category. In last year's article about mobile marketing, I mentioned how Google offers a way for you to build a free mobile website for a year. Google's GoMo can also take your existing website through a free diagnostic test to determine to what extent it is (or is not) already mobile friendly.

4) Google Alerts

At ProfitBooks, we have to constantly keep ourselves updated with latest developments in the accounting and taxation industry. For this, we rely on Google Alerts. This very useful free service from Google sends you an email alert whenever there is any news about topic of your interest appears on internet.
With this, you can stay updated about your industry news and can even track your competitor! For example, you can sign up to get notified whenever someone mentions your company, products, executives, or your competition. Its super simple to use – you just need to add a topic or a search phrase and create an alert.

5) Google Trends

You're in the process of evolving your business with the changing times. You need to determine what kind of marketing language and descriptive terminology to use for your sales materials, website copy, and search engine optimization. Consider plugging some of your terms into the Google Trends search bar to see how searches for these terms have changed over time. Look for those still trending upward, and review the additional detail Google provides.
In addition to the Google Alerts, Google Trends can be a great tool for helping you monitor industry trends. It enables you to evaluate the popularity of certain terms, compare them against other keyword variations, analyze how their popularity varies over time and in different regions/languages, and shows related keywords, which can be helpful in getting new keyword suggestions.

6) Google Voice

In an era when people use their phones to surf the web, it's only natural to start using the web to manage our phones. Google Voice, albeit only available in the U.S., allows you to do just, making it easy to manage multiple phone lines, create personalized voicemail messages depending on who's calling, and easily transcribe voicemail messages, making it much easier to stay on top of a busy voicemail inbox. 
To learn more about the various features available with Google Voice, check out Google's support documentation, and watch the video overview below.

7) Google FeedBurner

Want to grow your reach? Then you should be allowing your visitors to subscribe to your website content, particularly your blog, using feeds. By setting up a Google FeedBurner account, your site visitors can subscribe to your content and receive regular updates via their web browsers, RSS readers, or email. And considering subscribers are extremely critical to the growth and reach of a business blog, offering subscription options for your content isn't something you want to overlook. 

8) Public Data Explorer

Google’s Public Data Explorer provides public data and forecasts from a range of international organizations and academic institutions including the World Bank, OECD, Eurostat and the University of Denver. These can be displayed as line graphs, bar graphs, cross sectional plots or on maps.

9) Keyword Planner

If you are planning to start advertising on Google, Keyword Planner will give you an estimate of search traffic and budget. Its a great tool to find out which keywords people are searching for more often. You can slice and dice the data based on geography, gender, interest, browser, mobile device and much more.

10) Google Scholar

Fed up of routine articles on a specific topic – like business growth? Get more meaningful information using Google Scholar. It is an online, freely accessible search engine that provides a simple way to broadly search for scholarly literature. It searches a wide variety of sources, including academic publishers, universities, articles, theses, books, abstracts and court opinions.
Google Scholar aims to rank documents the way researchers do, weighing the full text of each document, where it was published, who it was written by, as well as how often and how recently it has been cited in other scholarly literature.

Thursday, 19 October 2017

Major Google’s limits you may not know exist!! - SEO , Explained.

10/19/2017 10:28:00 am
Google has a lot of different tools, and while they handle massive amounts of data, even Google has its limits. Here are some of the limits you may eventually run into.

1. 1,000 properties in Google Search Console

Per Google’s Search Console Help documentation, “You can add up to 1,000 properties (websites or mobile apps) to your Search Console account.”

2. 1,000 rows in Google Search Console

Many of the data reports within Google Search Console are limited to 1,000 rows in the interface, but you can usually download more. That’s not true of all of the reports, however (like the HTML improvements section, which doesn’t seem to have that limit).

3. Google Search Console will show up to 200 site maps


The limit for the number submitted is higher, but you will only be shown 200. Each of those could be an index file as well, which seems to have a display limit of 400 site maps in each. You could technically add each page of a website in its own site map file and bundle those into site map index files and be able to see the individual indexation of 80,000 pages in each property… not that I recommend this.

4. Disavow file size has a limit of 2MB and 100,000 URLs


According to Search Engine Roundtable, this is one of the errors that you can receive when submitting a disavow file.
5. Render in Google Search Console cuts off at 10,000 pixels

Google Webmaster Trends Analyst John Mueller had mentioned that there was a cutoff for the “Fetch as Google” feature, and it looks like that cutoff is 10,000 pixels, based on testing.

6. Google My Business allows 100 characters in a business name.

7. 10 million hits per month per property in GA (Google Analytics)

Once you’ve reached this limit, you’ll either be sampled or have to upgrade.

8. Robots.txt max size is 500KB


As stated on Google’s Robots.txt Specifications page, “A maximum file size may be enforced per crawler. Content which is after the maximum file size may be ignored. Google currently enforces a size limit of 500 kilobytes (KB).”

9. Sitemaps are limited to 50MB (uncompressed) and 50,000 URLs



All formats limit a single sitemap to 50MB (uncompressed) and 50,000 URLs. If you have a larger file or more URLs, you will have to break it into multiple sitemaps. You can optionally create a sitemap index file (a file that points to a list of sitemaps) and submit that single index file to Google. You can submit multiple sitemaps and/or sitemap index files to Google.

10. Keep URLs to 2,083 or fewer characters


While Google doesn’t have a limit, you probably shouldn’t go over Internet Explorer’s limit of 2,083 characters in the URL.

11. Google’s crawl limit per page is a couple hundred MBs


That is according to Google’s John Mueller and represents a significant jump from the 10MB limit in 2015.

12. Keep the number of links on a page to a few thousand at most


While Google doesn’t have a hard limit on the number of links per page, they do recommendkeeping it to “a reasonable number,” clarifying that this number is “a few thousand at most.”

13. 5 redirect hops at one time


Google’s John Mueller has said that Googlebot will follow up to five redirects at the same time. I don’t know if anyone has ever looked into the total number Google will follow. I did a little digging in Google Search Console and found one page still showing links as “via intermediate links” with a 10-hop chain. Yes, the original still showed in that case, but I also found some others that were cut off at six hops, even though they had more in the chain. I would say keep it to as few as you can, just in case.

14. No limit on word count on a page


It’s often recommended to keep it to 250 words, but there’s really no limit.

15. Google search limits to 32 words


Fun fact: Each word is also limited to 128 characters.

16. 16 words on alt text


While there’s not really a limit per se, this test is still live, and only the first 16 seem to count.
17. There is no limit to how many times a site can show on first page


That’s right, one domain can take the entire page if it’s relevant enough. 

18. YouTube maximum upload size is 128 GB or 12 hours


The maximum file size that you can upload is 128 GB or 12 hours, whichever is less. We’ve changed the limits on uploads in the past, so you may see older videos that are longer than 12 hours.

19. Google Keyword Planner limits you to 700


You are limited to 700 keywords in Keyword Ideas. This is also the limit when uploading a file to get search volume and trends, but you can upload 3,000 keywords at a time to the forecaster.

20. YouTube’s counter limit


YouTube’s counter used to be a 32-bit integer, limiting the possible video views it would show to a little over 2 billion (2,147,483,647). YouTube now uses a 64-bit integer, which can show ~9.22 quintillion views (9,223,372,036,854,775,808).
Google