Memcached: Difference between revisions

Line 1:

Memcached is a BSD licensed, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering. [https://github.com/memcached/memcached/wiki/TutorialCachingStory Story]

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering. See the [https://github.com/memcached/memcached/wiki/TutorialCachingStory Caching Story] that explains how every web application eventually turns to caching to increase performance.

* [https://github.com/memcached/memcached/wiki/Commands Commands]

See Also:

* MediaWiki has a ~~client~~ tool in the 'maintenance' directory: mcc.php

*[https://github.com/memcached/memcached/wiki/Commands Memcache Commands]

* [https://stackoverflow.com/questions/8420776/how-do-i-view-the-data-in-memcache Viewing data in memcache]

*MediaWiki has a diagnostic tool for interacting with memcached in the 'maintenance' directory: [https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/master/maintenance/mcc.php mcc.php]

*[https://stackoverflow.com/questions/8420776/how-do-i-view-the-data-in-memcache Viewing data in memcache]

<blockquote>"Cache is King" - borrowed from the aphorism "Cash is King" For a full exploration of object caching in MediaWiki see https://www.mediawiki.org/wiki/Object_cache</blockquote>

== Making cache scalable ==

MediaWiki connects to memcached through a proxy called McRouter (Mick-ROW-ter) <ref>https://wikitech.wikimedia.org/wiki/Memcached_for_MediaWiki#Mcrouter</ref> Mcrouter was developed by Meta engineering back in 2014 for scaling memcached deployments <ref>https://engineering.fb.com/2014/09/15/web/introducing-mcrouter-a-memcached-protocol-router-for-scaling-memcached-deployments/</ref>. It's a core component of cache infrastructure at Facebook and Instagram where mcrouter handles almost 5 billion requests per second at peak.

Because the routing and feature logic are abstracted from the client in mcrouter deployments, the client may simply communicate with destination hosts through mcrouter over a TCP connection using standard memcached protocol. Typically, little or no client modification is needed to use mcrouter, which was designed to be a drop-in proxy between the client and memcached hosts.

Mcrouter supports typical memcache protocol commands like get, set, delete, etc. and specific commands to access stats, version and so on.<ref>https://github.com/facebook/mcrouter/wiki</ref>

== MediaWiki Interfaces ==

'''WANObjectCache''' (or '''WANCache''') is the primary interface in MediaWiki for interacting with Memcached and mcrouter. As the name implies, WANObjectCache is a multi-datacenter aware caching interface. See the [https://doc.wikimedia.org/mediawiki-core/master/php/classWANObjectCache.html#details class docs] for more detail on using and deploying WANObjectCache. [https://gerrit.wikimedia.org/g/mediawiki/core/%2B/master/includes/libs/objectcache/ Source in Gerrit] includes a README describing stats that go into StatsD

<blockquote>Aside: WANObjectCache can also be deployed using [https://github.com/Netflix/dynomite Dynomite] as the intermediary - to use Amazon's Dynamo cache in a distributed way. See the [https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf 2007 whitepaper] for details on Amazon's Dynamo.

</blockquote>

If you are a MediaWiki developer - especially a front-end dev, you've probably seen references in code to the 'BagOStuff' and wondered "what the heck is it?" BagOStuff is an abstract Class representing a cache/ephemeral data store. Concrete classes then build upon or implement BagOStuff in ways that are compatible with those concrete storage mechanisms. <ref>https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/master/includes/libs/objectcache/BagOStuff.php</ref> WANCache builds on top of '''BagOStuff''', which is the lower level key-value interface to Memcached and other storage backends.

[[File:Wikipedia_Memcached_flow_2022.png]]

[[Category:Caching]]

[[Category:Performance]]

@@ Line 1: / Line 1: @@
 Memcached is a BSD licensed, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.
-Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering. [https://github.com/memcached/memcached/wiki/TutorialCachingStory Story]
+Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering. See the [https://github.com/memcached/memcached/wiki/TutorialCachingStory Caching Story] that explains how every web application eventually turns to caching to increase performance.
-* [https://github.com/memcached/memcached/wiki/Commands Commands]
+See Also:
-* MediaWiki has a client tool in the 'maintenance' directory: mcc.php
+*[https://github.com/memcached/memcached/wiki/Commands Memcache Commands]
-* [https://stackoverflow.com/questions/8420776/how-do-i-view-the-data-in-memcache Viewing data in memcache]
+*MediaWiki has a diagnostic tool for interacting with memcached in the 'maintenance' directory: [https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/master/maintenance/mcc.php mcc.php]
+*[https://stackoverflow.com/questions/8420776/how-do-i-view-the-data-in-memcache Viewing data in memcache]
+<blockquote>"Cache is King" - borrowed from the aphorism "Cash is King" For a full exploration of object caching in MediaWiki see https://www.mediawiki.org/wiki/Object_cache</blockquote>
+== Making cache scalable ==
+MediaWiki connects to memcached through a proxy called McRouter (Mick-ROW-ter) <ref>https://wikitech.wikimedia.org/wiki/Memcached_for_MediaWiki#Mcrouter</ref> Mcrouter was developed by Meta engineering back in 2014 for scaling memcached deployments <ref>https://engineering.fb.com/2014/09/15/web/introducing-mcrouter-a-memcached-protocol-router-for-scaling-memcached-deployments/</ref>. It's a core component of cache infrastructure at Facebook and Instagram where mcrouter handles almost 5 billion requests per second at peak.
+Because the routing and feature logic are abstracted from the client in mcrouter deployments, the client may simply communicate with destination hosts through mcrouter over a TCP connection using standard memcached protocol. Typically, little or no client modification is needed to use mcrouter, which was designed to be a drop-in proxy between the client and memcached hosts.
+Mcrouter supports typical memcache protocol commands like get, set, delete, etc. and specific commands to access stats, version and so on.<ref>https://github.com/facebook/mcrouter/wiki</ref>
+== MediaWiki Interfaces ==
+'''WANObjectCache''' (or '''WANCache''') is the primary interface in MediaWiki for interacting with Memcached and mcrouter. As the name implies, WANObjectCache is a multi-datacenter aware caching interface. See the [https://doc.wikimedia.org/mediawiki-core/master/php/classWANObjectCache.html#details class docs] for more detail on using and deploying WANObjectCache. [https://gerrit.wikimedia.org/g/mediawiki/core/%2B/master/includes/libs/objectcache/ Source in Gerrit] includes a README describing stats that go into StatsD
+<blockquote>Aside: WANObjectCache can also be deployed using [https://github.com/Netflix/dynomite Dynomite] as the intermediary - to use Amazon's Dynamo cache in a distributed way. See the [https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf 2007 whitepaper] for details on Amazon's Dynamo.
+</blockquote>
+If you are a MediaWiki developer - especially a front-end dev, you've probably seen references in code to the 'BagOStuff' and wondered "what the heck is it?" BagOStuff is an abstract Class representing a cache/ephemeral data store. Concrete classes then build upon or  implement BagOStuff in ways that are compatible with those concrete storage mechanisms.   <ref>https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/master/includes/libs/objectcache/BagOStuff.php</ref> WANCache builds on top of '''BagOStuff''', which is the lower level key-value interface to Memcached and other storage backends.
+[[File:Wikipedia_Memcached_flow_2022.png]]
+[[Category:Caching]]
+[[Category:Performance]]