|
|
| Line 3: |
Line 3: |
| | Find out "What's that Wiki Running?" at https://freephile.org/wikireport | | Find out "What's that Wiki Running?" at https://freephile.org/wikireport |
| | | | |
| − | == README ==
| |
| | | | |
| − | Using the MediaWiki API, we want to query a wiki installation about the | + | == APIs in action == |
| − | site's metadata that tells us the version, and the extensions running there.
| + | This project is about two APIs. Using them, and having them work together. We're using the MediaWiki API, and and the CiviCRM API. |
| | | | |
| − | We want to be able to retrieve this data, and then import it into our [[CiviCRM]] database
| + | == MediaWiki == |
| − | We also want to be able to create a nice public-facing reporting tool that we
| |
| − | can use to do one-off reports, or to show to site owners.
| |
| | | | |
| − | So we'll develop it in 3 phases:
| + | We're using the MediaWiki API for a couple different purposes. |
| − | <ol>
| + | Most obviously we're using the API, which is available in every MediaWiki installation, to query that installation's exact setup, statistics and extensions. Secondly, we're using the MediaWiki API to re-use content. We're showing an example of using MediaWiki as a back-end content editing platform, while the '''Wiki Report''' project publishes that content with 5 simple lines of code on a completely different website. Through the API, we could re-use any MediaWiki content on any other project or platform. |
| − | <li> The public-facing reporting tool will be developed first
| |
| − | <li> The conduit to read and write data to the CiviCRM system
| |
| − | <li> Publicize the reporting tool <br />
| |
| − | <ol>
| |
| − | <li>through campaigns to the people we have in the database
| |
| − | <li>through social and other networks
| |
| − | <li>possibly as a case study or info example on how to make APIs work for you and talk to each other.
| |
| − | </ol>
| |
| − | </ol>
| |
| − | The goal of publicizing the reporting tool is to make it an inbound marketing tool
| |
| − | If we can collect information about wikis that we don't know about, then we can market to those people.
| |
| | | | |
| | + | == CiviCRM == |
| | | | |
| − | To create the UI, we'll use Bootstrap
| + | The free '''Wiki Report''' generates a lot of data about wikis across the Internet. We collect and/or report on this data by storing it in a CiviCRM database. |
| − | * see https://en.wikipedia.org/wiki/Bootstrap_%28front-end_framework%29
| |
| − | * see http://getbootstrap.com/getting-started/#examples
| |
| − | * see https://freephile.org/wiki/Bootstrap
| |
| | | | |
| − | For some wikis, we may need to determine the api URL when we only have the "wiki" URL.
| + | == Bootstrap == |
| − | Using the common setup where the code is hosted at w/ and the pretty URLs are at wiki/, we
| |
| − | find this in the source of the Main_Page:
| |
| − | <nowiki><link rel="EditURI" type="application/rsd+xml" href="https://freephile.org/w/api.php?action=rsd" /></nowiki>
| |
| | | | |
| − | The MediaWiki API supports many formats, with json and jsonfm the most useful. json is what you think it is,
| + | We're using Bootstrap for building a nice HTML5, responsive and mobile-first website. |
| − | and jsonfm is formatted for viewing in the browser (good for development only)
| |
| | | | |
| − | The API has many possible parameters. https://www.mediawiki.org/wiki/API:Siteinfo
| + | == Development == |
| − | The default is 'general', and you should combine as many as you want. So, to get
| |
| − | all the info that we're after, we would compose a query like so:
| |
| − | <nowiki>https://freephile.org/w/api.php?action=query&meta=siteinfo&format=jsonfm&siprop=general|extensions|statistics</nowiki>
| |
| | | | |
| − | == General Info ==
| + | For details, see [[Wiki report/development]] |
| − | From 'General' we're interested in
| |
| − | * "base" (which we already have? chicken/egg)
| |
| − | * "sitename"
| |
| − | * "logo"
| |
| − | * "generator"
| |
| − | * "phpversion"
| |
| − | * "phpsapi"
| |
| − | * "dbtype"
| |
| − | * "dbversion"
| |
| − | * "lang" --just in case it's not 'en'
| |
| − | * "timezone"
| |
| − | * "time"
| |
| − | * "favicon"
| |
| − | | |
| − | I've found that there can be empty values; and which ones are empty depend on the instance, so we won't
| |
| − | hard-code what is in the report... we'll just report on what we find. Likewise, we'll endeavor to collect all the info
| |
| − | that is useful, but there will be information that is unavailable in some cases.
| |
| − | | |
| − | == Exensions ==
| |
| − | After the general info, we are especially interested in
| |
| − | extensions. Again, the info available for any given extension is going to vary, so we'll report on what's available,
| |
| − | and likewise we will record what we can find.
| |
| − | | |
| − | Extensions is an array of items. An extension item will look like this:
| |
| − | <source lang=javascript>
| |
| − | {
| |
| − | "query": {
| |
| − | "extensions": [
| |
| − | | |
| − | {
| |
| − | "type": "other",
| |
| − | "name": "Html2Wiki",
| |
| − | "descriptionmsg": "html2wiki-desc",
| |
| − | "author": "Greg Rundlett",
| |
| − | "url": "https://www.mediawiki.org/wiki/Extension:Html2Wiki",
| |
| − | "version": "2015.02",
| |
| − | "vcs-system": "git",
| |
| − | "vcs-version": "c24896064a6a604f71f7e3253373a59d04fe19bc",
| |
| − | "vcs-url": false,
| |
| − | "vcs-date": "2015-04-28T17:41:26Z",
| |
| − | "license-name": "GPL-2.0+",
| |
| − | "license": "/wiki/Special:Version/License/Html2Wiki"
| |
| − | }
| |
| − | | |
| − | | |
| − | ]
| |
| − | }
| |
| − | }
| |
| − | | |
| − | </source>
| |
| − | | |
| − |
| |
| − | Do we timestamp in the CiviCRM database? The "profile" will change over time, and so do we care about what it used to be?
| |
| − |
| |
| − | == Other Siprops ==
| |
| − | siprop=statistics is included in the report
| |
| − | <source lang="javascript">
| |
| − | {
| |
| − | "query": {
| |
| − | "statistics": {
| |
| − | "pages": 1363,
| |
| − | "articles": 198,
| |
| − | "edits": 5459,
| |
| − | "images": 963,
| |
| − | "users": 6,
| |
| − | "activeusers": 1,
| |
| − | "admins": 2,
| |
| − | "jobs": 3660
| |
| − | }
| |
| − | }
| |
| − | }
| |
| − | </source>
| |
| − | | |
| − | <code>siprop=usergroups</code> is interesting if a wiki is doing anything with groups
| |
| − | | |
| − | | |
| − | <code>api.php?action=query&meta=siteinfo&format=jsonfm&siprop=rightsinfo</code>
| |
| − | gives the copyright url and text
| |
| − | | |
| − | <code>api.php?action=query&meta=siteinfo&format=jsonfm&siprop=namespaces|namespacealiases</code>
| |
| − | gives the namespaces and aliases, which can reveal 'private' namespaces
| |
| − | | |
| − | <code>api.php?action=query&meta=siteinfo&format=jsonfm&siprop=fileextensions</code>
| |
| − | shows you the file extensions allows for file upload.
| |
| − | | |
| − | <code>siprop=libraries</code> is kind of interesting in that you probably didn't know you ran that in your wiki
| |
| − | | |
| − | <code>api.php?action=query&meta=siteinfo&format=jsonfm&siprop=showhooks</code>
| |
| − | is really interesting because it shows you what code is listening to what hooks.
| |
| − | | |
| − | <code>api.php?action=query&meta=siteinfo&format=jsonfm&siprop=extensiontags</code>
| |
| − | is useful as a documentation page that shows authors the additional tags usable on this wiki
| |
| − | | |
| − | == Background ==
| |
| − | | |
| − | The MediaWiki <abbr title="Application Programming Interface">[[API]]</abbr> offers great detail into what a [[MediaWiki]] wiki is running.
| |
| − | | |
| − | The ApiSandbox extension offers a GUI way to explore and even operate on the API <ref>[https://git.wikimedia.org/blob/mediawiki%2Fextensions%2FApiSandbox/c69d2923bd5ff5424ce89a12b87f02708c80d487/SpecialApiSandbox.php code]</ref>
| |
| − | | |
| − | The most popular data format for API communication is <abbr title="JavaScript Object Notation">[http://www.json.org/ JSON]</abbr>.
| |
| − | | |
| − | [https://php.net/manual/en/intro.json.php PHP handles JSON] consistent with the expanded definition of "JSON text" in the newer [http://www.faqs.org/rfcs/rfc7159.html RFC 7159].
| |
| − | | |
| − | What I need to do is collect data, based on some data that I have in a CiviCRM comment (and the original csv file that was imported to CiviCRM), and re-import or add the new data to CiviCRM for market segmentation and intelligence.
| |
| − | | |
| − | I want to report on that data internally to see my best potential prospects.
| |
| − | | |
| − | I want to include that data in marketing to show prospects that they need to upgrade. Or that they have a number of extensions. Including large amounts of data in a marketing message would be best in the form of a "free report'. The reporting interface can also be used as an ad-hoc intelligence reporter and collector.
| |
| − | | |
| − | Is there an API for inserting data into CiviCRM? (Yes CiviCRM has a full API) Or, do I need to rely on the import tools and formatting my CSV with the correct external ID so as to avoid duplicates?
| |
| − | | |
| − | The basic UI for What's that Wiki Running? could be handled by [[jQuery]], but with [[Bootstrap]] added in, we get a bit more stylesheet. Still, do we need that? [https://api.jquery.com/category/ajax/global-ajax-event-handlers/ jQuery can handle the AJAX] Simple [http://www.zachhunter.com/2010/04/json-objects-to-html-table/ JavaScript could handle the data presentation] (and jQuery probably has some useful methods) and we can add [https://jqueryui.com/ jQuery UI] for advanced interactions or widgets.
| |
| − | | |
| − | To do cross-domain javascript requests, we'll need jsonp but, it turns out that you just can't make a secure AJAX UI to insecure web content (Duh!).
| |
| − | | |
| − | In the jQuery <code>.ajax</code> method, there are several settings. The 'error' setting is a callback, and we could write a function there to try other variations on the domain to find the API endpoint.
| |
| − | | |
| − | Interesting example: http://en.banglapedia.org/index.php?title=Special:Version which uses [https://www.mediawiki.org/wiki/Skin:MediaWikiBootstrap the "MediaWiki Bootstrap" skin] See http://www.mediawikibootstrapskin.co.uk/index.php?title=Main_Page
| |
| − | | |
| − | I decided to use [[Bootstrap]] for this project. At first, I was tempted to build a fully client-side framework and JavaScript solution. Since CiviCRM has an API, I could potentially even use JavaScript for pushing data into the backend (authenticated with a secret). Ultimately, Bootstrap was used on the client-side, but PHP was used to create the backend.
| |
| − | | |
| − | == Data Table systems in the UI ==
| |
| − | | |
| − | [https://datatables.net/examples/styling/bootstrap.html DataTables can integrate seamlessly with Bootstrap], but it's unlikely that I will need the robustness of [[Data Tables]] for this effort. If I wanted to show many records, then yes. But for now, I want to focus on showing the particular details of a specific wiki site.
| |
| − | | |
| − | DynaTables is a jQuery plugin that I began prototyping with, but again, it's not something that I will need since I don't have large datasets to manipulate. I just want to focus on a single wiki at a time. [http://www.dynatable.com/#existing-json With existing JSON], you still have to know the 'layout' of your data and setup a blank table with the correct structure. I'm more interested in a generic function that you could throw any JSON at, and it would spit out a table.
| |
| − | | |
| − | [https://labs.mbrooks.info/demos/Tidy-Table/ Tidy Table] is a jQuery plugin that creates HTML tables from the data that you feed into the client as JSON The only interesting thing here is the fact that I might want to have something akin to their demo for the initial form state: Allow the user to quickly/easily select which parts of the MW API they want to report on.
| |
| − | | |
| − | == Code ==
| |
| − | You can clone, fork, and send pull requests at github https://github.com/freephile/wikireport
| |
| − | | |
| − | {{References}}
| |
