Open main menu
Freephile Wiki

CiviCRM

Revision as of 12:34, 3 December 2018 by Freephile (talk | contribs) (extract upgrade)

(diff) ← Older revision | Approved revision (diff) | Latest revision (diff) | Newer revision → (diff)

Subpages

There are several articles that delve into the specific aspects of CiviCRM

Documentation

Developer documentation is found in the (Atlassian Confluence) wiki[1]

The end-user and administrator docs for Civi are compiled into 'Books' which you can find at https://book.civicrm.org/ However, the underlying documentation system, tooling and workflows are changing (from FLOSS Manuals[2] 'booki' system to GitBook[3] and Read The Docs is still under consideration). Thus, the newest documentation is available (temporarily) at http://gitbook.civicrm.org/. You can contribute to the documentation at https://github.com/civicrm/civicrm-user-guide. Ultimately the latest docs (and system) will be found at https://book.civicrm.org/

Learn

Go to CiviTeacher.com to learn about using CiviCRM

CiviTeacher.com is a place for high quality videos on CiviCRM CiviDesk is a CiviCRM consultant offering extensive training courses.

Help

There is help in the 'forums' But the CiviCRM Community Forums have now been deprecated by a new StackExchange https://civicrm.stackexchange.com/

Installation

CiviCRM sits on top of a CMS: either Drupal, WordPress, or Joomla!.

The CiviCRM User and Administrator Book and also the wiki (see https://civicrm.org/go/books) are invaluable in getting setup.

When installing on top of Drupal, if you have the HTMLPurifier module enabled, you will run into a conflict that you can resolve by disabling the module in Drupal proper, or else modifying the bootstrap code in civicrm [4]

Also, the default theme (and most other themes?) don't work well for CiviCRM - especially the admin side. So, drush dl civi_bartik and enable it for the admin side.

After installation, there is a checklist that you should complete (e.g. http://example.org/civicrm/admin/configtask)

Installation Checklist[5]:

  1. Enable the CiviBartik theme, for Civi admin, and then immediately configure various blocks to NOT appear in that theme (remove everything from column two, so that you get a wide display)
  2. enable the components
  3. check/enable permissions (Drupal)
  4. disable the headers and footers for mailings (place mandatory tokens in your templates instead)
  5. set the message template for mailings (note that you'll want to design, and create all assets for your mail templates, and host them)
  6. setup custom fieldsets and data fields. Before you do this, learn about Option Groups
  7. map import data to fields, groups, tags. Also do a large amount of data wrangling (normalization; and ETL) to get source data in a format suitable for use with CiviCRM. The data that I have is "composite" because it has individuals embedded inside organization records. So I needed to extract and flatten out contact records from their organization records; plus create and maintain a simple "external ID" system to relate them after import. This means parsing, slicing and dicing, string manipulation and making corrections or formatting data a particular way. I almost installed Pentaho Data Integration (kettle) because that tool is (supposedly) built for this job. [6]
  8. Be sure to tag each import with some unique tag. You can always remove those tags later (easily). When you inevitably run into problems during import, the proper way to delete records is to search by tag, and delete.
  9. Option Groups. There are many things in CiviCRM that are already configured as "Option Groups". "Website" is one example. When adding an Organization or Contact, and you want to enter data about their website, it could be one of many types: main, work, personal, facebook, twitter, pinterest, github etc. These are defined in the option group for "website". You can modify these to suit your data and your needs. As another example, "Campaign Type" comes defined as 'Direct Mail', 'Referral Program', and 'Customer Engangement'. I added 'Marketing' so that I can do a generic (email/web) marketing campaign.
  10. If you plan to use the CiviCase component to manage the common constituent "projects" or "workflows" and their associated timelines, then you'll need to create your own "Case Types". Look at the existing "Case Types" for reference.
  11. Gedit.svg todo review docs and setup cron to do things like geocoding. Your mail campaigns will not send without cron Drupal cron is best managed with drush
  12. Test and set your SPF record for your domain so that you can use Mailer, and review the docs for email system configuration I was unable to get CiviCRM to use Google's smtp.gmail.com server, nor relay-smtp.gmail.com, even with an IP address whitelisted. This is because Digital Ocean is still dropping all outbound SMTP traffic at their firewall (telnet doesn't even connect). Somehow, if I smarthost it through Postfix it actually works. I want to use Google for delivery because using mail() from an IP at Digital Ocean will result in mail being flagged as spam or silently dropped by several major providers (e.g. Yahoo, Microsoft). Besides, if I'm paying for GAFYD, then I want to actually use it! Google IS pretty well known for their email delivery capability! Ultimately, Google is not a good choice for delivery. Use a vendor. (See CiviCRM/CiviMail and Email Marketing
  13. Turn on logging in the Administration console, otherwise each record has a changelog, but there is no detail in the log!
  14. Synchronize Users to Contacts
  15. Create a BOD group, and put each member into the group


Interestingly, CiviCRM optionally uses wkhtmltopdf to convert HTML to PDF


Upgrading

CiviCRM/upgrade

Debugging and the CV command-line

If you have your CiviCRM sending errors to the Drupal Watchdog Log (CiviCRM Administer > System Settings > Debugging and Error Handling), you can simply navigate to it in the Drupal Admin (Reports > Recent log messages)

There are many tools you can use to debug your CiviCRM instance. One of these is the cv tool available on Github. 

cv version v0.1.27

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display this help message
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands:
  api                     Call an API
  cli                     Load interactive command line
  dis                     Disable an extension
  dl                      Download and enable an extension
  en                      Enable an extension
  ev                      Evaluate a snippet of PHP code
  flush                   Flush system caches
  help                    Displays help for a command
  list                    Lists commands
  path                    Look up the path to a file or directory
  scr                     Execute a PHP script
  url                     Compose a URL to a CiviCRM page
 ang
  ang:html:list           List Angular HTML files
  ang:html:show           Show an Angular HTML file
  ang:module:list         List Angular modules
 api
  api:batch               Call an API (batch mode)
 debug
  debug:container         Dump the container configuration
  debug:event-dispatcher  Dump the list of event listeners
 ext
  ext:disable             Disable an extension
  ext:download            Download and enable an extension
  ext:enable              Enable an extension
  ext:list                List extensions
  ext:uninstall           Uninstall an extension and purge its data
  ext:upgrade-db          Apply DB upgrades for any extensions
 php
  php:boot                Generate PHP bootstrap code
  php:eval                Evaluate a snippet of PHP code
  php:script              Execute a PHP script
 vars
  vars:fill               Generate a configuration file for any missing site data
  vars:show               Show the configuration of the local CiviCRM installation

Also, don't forget to use Drush if you're on Drupal; the CiviCRM API; and the ConfigAndLog directory log. The ConfigAndLog directory is in a path like ./sites/default/files/civicrm/ConfigAndLog

Profiles

You can create form sets called "Profiles" to be able to easily collect info through the exact forms you need. CiviCRM Profiles allow you to aggregate groups of fields and include them in your site as input forms, contact display pages, and search and listings features. They provide a powerful set of tools for you to collect information from constituents and selectively share contact information.

Custom Data

You can import custom data into 'fields', but first you need to create a fieldset (at civicrm/admin/custom/group?reset=1) to contain even a single custom field. (Note that fieldsets are referred to as 'custom groups' in the code/system, but should not be confused with contact groups which is another thing altogether.) Some (most?) data such as health info (weight, cholesterol, height) or volunteer experience (date, location, volunteer activity) or hike (date, trail, notes) should be multi-value since you'll want to record more than a single fieldset per contact. A problem that I ran into was that a multi-value fieldset does not show up in the list of available tokens within the CiviMail component. These tokens are still available, but you have to write them into the email message by hand. On the other hand, if you create a single value custom data field, those do appear in CiviMail. So, if you create group "work experience", and fields "Job Title", "Employer", "Date Start", "Date End" etc, AND in the setup of that group, you __do not__ check the box that says "Does this Custom Field Set allow multiple records?", you'll end up with a single value field group and those values will be represented by tokens in CiviMail. Once created, you cannot change the type of a Field Set from multi to single or single to multi.

In short, there are all kinds of problems with Custom Data. It's inconsistently represented across Forms, Reports, Contacts, CiviMail. Plus, even when you spend the time to create a Profile display of the custom data, it's almost useless. The paging is non-existant; table view with sorting is non-existant. In Reports (where you can specify custom data fields for output), oddly the first value is displayed rather than the latest.

Multi-value Custom Data is such a problem that there is even a wizard for importing it. civicrm/import/custom?reset=1


Custom tokens (based on custom data) can be added for organizations as well. These tokens will not be displayed in the list of available tokens, but can be added manually. The format is {contact.custom_12} � where 12 is the ID of the custom data field. To find the custom data field ID, go Administer > Customize Data & Screens > Custom Fields and click �edit� on the field you want to use. Look at the URL. The last part of the URL will be an equal sign and a number (=12). The number (12 in this example) is the id of that custom field.

You can create your own tokens by implementing hook_civicrm_tokens() and hook_civicrm_tokenValues(). See 

<?php
/**
 * A convenience function so that we can map our custom fields and labels
 * in one place, and share them between hooks without having to write to the db
 * with variable_set()
 */
function getHaystack($stack='general') {
    $genKeys = array (
        "wUrl"       => "custom_40",
        "mainpage"   => "custom_41",
        "base"       => "custom_42",
        "sitename"   => "custom_43",
        "logo"       => "custom_44",
        "generator"  => "custom_45",
        "phpversion" => "custom_46",
    // "phpsapi", 
        "dbtype"     => "custom_47",
        "dbversion"  => "custom_48",
    // "externalimages", 
    // "langconversion", 
    // "titleconversion", 
    // "linkprefixcharset", 
    // "linkprefix", 
    // "linktrail", 
    // "legaltitlechars", 
    // "git-hash", 
    // "git-branch", 
    // "case", 
    // "lang", 
    // "fallback", 
    // "fallback8bitEncoding", 
        "writeapi"    => "custom_49",
        "timezone"    => "custom_50",
        "timeoffset"  => "custom_51",
        "articlepath" => "custom_52",
        "scriptpath"  => "custom_53",
    // "script", 
    // "variantarticlepath", 
        "server"      => "custom_54",
        "servername"  => "custom_55",
        "wikiid"      => "custom_56",
        "time"        => "custom_57",
        "maxuploadsize" => "custom_58",
    // "thumblimits", 
    // "imagelimits", 
        "favicon"     => "custom_59",
    );
    $statKeys = array(
        "wUrl"     => "custom_60",
        "pages"    => "custom_61",
        "articles" => "custom_62",
        "edits"    => "custom_63",
        "images"   => "custom_64",
        "users"    => "custom_65",
        "activeusers" => "custom_66",
        "admins"   => "custom_67",
        "jobs"     => "custom_68",
    );
    switch ($stack) {
        case 'general':
            $haystack = $genKeys;
            $addLabels = array (
                'recorded' => 'custom_69',
            );
            $haystack += $addLabels;
            break;
        case 'stats':
            $haystack = $statKeys;
            $addLabels = array (
                'recorded' => 'custom_70',
            );
            $haystack += $addLabels;
            break;
        default:
            die('no stack by that name');
    }
    return $haystack;
}
/**
 * implementation of hook_civicrm_tokens()
 * Much appreciation and thanks to Eileen McNaughton who helped me get the logic
 * correct on this. cf.https://github.com/eileenmcnaughton/civicrm_views_token/blob/master/civicrm_views_token.module#L16
 * 
 * In the end, we want to populate more $tokens in a format like
 * $token['general.generator'] = 'Generator';
 * $token['general.sitename'] = 'Sitename';
 * where the $token key is the element that needs to be populated 
 * by hook_civicrm_tokenValues()
 * The $token value here is used in the UI as a label for the token.
 */
function eqt_civicrm_tokens(&$tokens) {
    
    $labels = getHaystack('general');
    $tokens['general'] = array();
    foreach ($labels as $k => $v) {
        $tokens['general']["general.$k"] = "$k ($v)";
    }
   
    $labels = getHaystack('stats');
    $tokens['stats'] = array();
    foreach ($labels as $k => $v) {
        $tokens['stats']["stats.$k"] = "$k ($v)";
    }
}
/**
 * implementation of hook_civicrm_tokenValues()
 */
function eqt_civicrm_tokenValues(&$values, $cids, $job = null, $tokens = array(), $context = null) {
    // for debugging
    //    watchdog('eqt',"eqt_civicrm_tokenValues(\$values, \$cids, \$job, \$tokens, \$context)<pre>\n\$values=" . var_export($values,1) . "\n\$cids=" . var_export($cids,1) . "\n\$job=" . $job ."\n\$tokens=" . var_export($tokens,1) . "\n\$context=" . $context . "\n</pre>");
    
    if ( array_key_exists('general', $tokens) ) {
        $haystack = getHaystack('general');
        foreach ($cids as $cid) {
            $params = array(
                'sequential' => 1,
                'entity_id' => $cid,
            );
            $result = civicrm_api3('CustomValue', 'get', $params);
            if (!$result['is_error']) {
                $customdata = ($result['values']);
                foreach ($customdata as $val) {
                    $needle = 'custom_' . $val['id'];
                    $label = array_search($needle, $haystack);
                    if (!$label) {
                        continue;
                    }
                    $label = 'general.' . $label;
                    $values[$cid][$label] = $val['latest'];
                }
            }
        }
    }
    
    if ( array_key_exists('stats', $tokens) ) {
        $haystack = getHaystack('stats');
        foreach ($cids as $cid) {
            $params = array(
                'sequential' => 1,
                'entity_id' => $cid,
            );
            $result = civicrm_api3('CustomValue', 'get', $params);
            if (!$result['is_error']) {
                $customdata = ($result['values']);
                foreach ($customdata as $val) {
                    $needle = 'custom_' . $val['id'];
                    $label = array_search($needle, $haystack);
                    if (!$label) {
                        continue;
                    }
                    $label = 'stats.' . $label;
                    $values[$cid][$label] = $val['latest'];
                }
            }
        }
    }
}

Custom Token extensions

Sarah Gladstone of pogstone contributed a couple different Token extensions that you'll find in the extensions system and on github https://github.com/sgladstone/com.pogstone.contenttokens/blob/master/contenttokens.php


Todo

  • Gedit.svg todo Install, and use overrides for DEV/PROD and make sure you can use the installation on multiple hosts [8]

Extensions

There are multiple ways to use the cv command to download and install extensions:

Download a published extension from the directory (long name).
cv dl org.example.foobar
Download a published extension from the directory (short name).
cv dl foobar
Download an unpublished extension (long name and zip URL)
cv dl org.example.foobar@http://example.org/files/foobar-1.2.zip
Download a pre-release (alpha/beta) from the directory.
cv dl --dev foobar


Developing Extensions

API

  1. Learn the Civi API (Go to /civicrm/api in your installation)
  2. Use the 'API Explorer' and the 'API Docs' interface that is available in your installation
  3. http://wiki.civicrm.org/confluence/display/CRMDOC40/CiviCRM+Public+APIs
  4. http://wiki.civicrm.org/confluence/display/CRMDOC/API+Examples
  5. http://wiki.civicrm.org/confluence/display/CRMDOC/Using+the+API
  6. http://wiki.civicrm.org/confluence/display/CRMDOC/API+Reference
  7. Warning old, probably incorrect: http://wiki.civicrm.org/confluence/display/CRMDOC32/Using+CiviCRM+APIs+-+Code+Snippets

Database

The database code in CiviCRM is divided into two logical sections: the DAO and the BAO. The BAO holds the "business logic" for objects and extends the DAO. The DAO is concerned with data to/from the database backend and it's definition (object to relational database mapping, aka "ORM"). Both are an extension of the PEAR DB DataObject class. As of August 2015, this is true and this blog post from 2006 gives some more (early) background. 

// from CRM/core/DAO.php

require_once 'PEAR.php';
require_once 'DB/DataObject.php'
class CRM_Core_DAO extends DB_DataObject

In 2009, Doctrine was proposed as a OODB approach instead of an ORM. There are other discussions too in the "Architecture Series" that are worth reviewing if you want to know more about database development in CiviCRM.

Notes

"Views" in Drupal (in core as of Drupal 8 [9]) is a good way to get reports from your data. If you want to use "Views" for your CiviCRM instance, you have to manually integrate it http://wiki.civicrm.org/confluence/display/CRMDOC/Views3+Integration Admin = civicrm/admin/setting/uf

References

  1. ↑ https://civicrm.org/improve-documentation
  2. ↑ http://www.flossmanuals.org/history
  3. ↑ https://www.gitbook.com/about
  4. ↑ http://forum.civicrm.org/index.php?topic=28429.0
  5. ↑ See also: https://hq.palantetech.coop/projects/commons/wiki/CiviCRM_launch_checklist
  6. ↑ But like since version 3, Pentaho still doesn't work even at 5. The install docs are missing. The thing failed to even load, with no error.
  7. ↑ GingerFeet is a Drupal and CiviCRM consultant + hosting provider like eQuality Technology
  8. ↑ http://wiki.civicrm.org/confluence/display/CRMDOC/Moving+an+Existing+Installation+to+a+New+Server+or+Location
  9. ↑ https://www.drupal.org/project/views
Retrieved from "https://wiki.freephile.org/wiki/index.php?title=CiviCRM&oldid=9286"