Difference between revisions of "CiviCRM"

From Freephile Wiki
Jump to navigation Jump to search
(extract upgrade)
(adds caution)
(19 intermediate revisions by the same user not shown)
Line 1: Line 1:
<!-- embed a video -->
+
For things specific to CiviCRM on WordPress, see the [[CiviCRM/WordPress]] article.
{{#ev:vimeo|192125770}}
 
  
== Subpages ==
+
Here we'll deal with the general plus Drupal-specific info (I think it's installed more with Drupal)
There are several articles that delve into the specific aspects of CiviCRM
 
{{#subpages:}}
 
 
 
== Documentation ==
 
Developer documentation is found [https://wiki.civicrm.org/confluence/display/CRM/CiviCRM+Wiki in the (Atlassian Confluence) wiki]<ref>https://civicrm.org/improve-documentation</ref>
 
 
 
The end-user and administrator docs for Civi are compiled into 'Books' which you can find at https://book.civicrm.org/  However, the underlying documentation system, tooling and workflows [https://civicrm.org/blogs/michael-mcandrew/moving-civicrms-user-and-administrator-guide-gitbook-or-readthedocs are changing] (from [https://flossmanuals.net/ FLOSS Manuals]<ref>http://www.flossmanuals.org/history</ref> 'booki' system to [https://www.gitbook.com/ GitBook]<ref>https://www.gitbook.com/about</ref> and [https://read-the-docs.readthedocs.org/en/latest/index.html Read The Docs] is still under consideration).  Thus, the newest documentation is available (temporarily) at http://gitbook.civicrm.org/. You can contribute to the documentation at https://github.com/civicrm/civicrm-user-guide. Ultimately the latest docs (and system) will be found at https://book.civicrm.org/
 
  
 
== Learn ==
 
== Learn ==
 
[[File:Applications-education.svg|thumb|link=http://civiteacher.com|Go to CiviTeacher.com to learn about using CiviCRM|200px]]
 
[[File:Applications-education.svg|thumb|link=http://civiteacher.com|Go to CiviTeacher.com to learn about using CiviCRM|200px]]
 
[http://civiteacher.com CiviTeacher.com] is a place for high quality videos on CiviCRM
 
[http://civiteacher.com CiviTeacher.com] is a place for high quality videos on CiviCRM
[https://www.cividesk.com CiviDesk] is a CiviCRM consultant offering extensive training courses.
 
  
 
== Help ==
 
== Help ==
Line 31: Line 22:
 
After installation, there is a checklist that you should complete (e.g. http://example.org/civicrm/admin/configtask)
 
After installation, there is a checklist that you should complete (e.g. http://example.org/civicrm/admin/configtask)
  
Installation Checklist<ref>See also: https://hq.palantetech.coop/projects/commons/wiki/CiviCRM_launch_checklist</ref>:
+
Some of the things needed for a first-time setup:
 
<ol>
 
<ol>
 
<li>Enable the CiviBartik theme, for Civi admin, and then immediately configure various blocks to NOT appear in that theme (remove everything from column two, so that you get a wide display)
 
<li>Enable the CiviBartik theme, for Civi admin, and then immediately configure various blocks to NOT appear in that theme (remove everything from column two, so that you get a wide display)
 
<li>enable the [http://book.civicrm.org/user/current/introduction/components/ components]
 
<li>enable the [http://book.civicrm.org/user/current/introduction/components/ components]
 
<li>check/enable permissions (Drupal)
 
<li>check/enable permissions (Drupal)
<li>disable the headers and footers for mailings (place mandatory tokens in your templates instead)
+
<li>set the headers and footers for mailings
<li>set the message template for mailings (note that you'll want to design, and create all assets for your mail [[templates]], and host them)
+
<li>set the message template for mailings
 
<li>setup custom fieldsets and data fields.  Before you do this, learn about Option Groups
 
<li>setup custom fieldsets and data fields.  Before you do this, learn about Option Groups
 
<li>map import data to fields, groups, tags.  Also do a large amount of data wrangling (normalization; and ETL) to get source data in a format suitable for use with CiviCRM.  The data that I have is "composite" because it has individuals embedded inside organization records.  So I needed to extract and flatten out contact records from their organization records; plus create and maintain a simple "external ID" system to relate them after import.  This means parsing, slicing and dicing, string manipulation and making corrections or formatting data a particular way.  I almost installed Pentaho Data Integration (kettle) because that tool is (supposedly) built for this job.  <ref>But like since version 3, Pentaho still doesn't work even at 5.  The install docs are missing.  The thing failed to even load, with no error.</ref>  
 
<li>map import data to fields, groups, tags.  Also do a large amount of data wrangling (normalization; and ETL) to get source data in a format suitable for use with CiviCRM.  The data that I have is "composite" because it has individuals embedded inside organization records.  So I needed to extract and flatten out contact records from their organization records; plus create and maintain a simple "external ID" system to relate them after import.  This means parsing, slicing and dicing, string manipulation and making corrections or formatting data a particular way.  I almost installed Pentaho Data Integration (kettle) because that tool is (supposedly) built for this job.  <ref>But like since version 3, Pentaho still doesn't work even at 5.  The install docs are missing.  The thing failed to even load, with no error.</ref>  
Line 43: Line 34:
 
<li>Option Groups.  There are many things in CiviCRM that are already configured as "Option Groups".  "Website" is one example.  When adding an Organization or Contact, and you want to enter data about their website, it could be one of many types: main, work, personal, facebook, twitter, pinterest, github etc.  These are defined in the option group for "website".  You can modify these to suit your data and your needs.  As another example, "Campaign Type" comes defined as 'Direct Mail', 'Referral Program', and 'Customer Engangement'.  I added 'Marketing' so that I can do a generic (email/web) marketing campaign.
 
<li>Option Groups.  There are many things in CiviCRM that are already configured as "Option Groups".  "Website" is one example.  When adding an Organization or Contact, and you want to enter data about their website, it could be one of many types: main, work, personal, facebook, twitter, pinterest, github etc.  These are defined in the option group for "website".  You can modify these to suit your data and your needs.  As another example, "Campaign Type" comes defined as 'Direct Mail', 'Referral Program', and 'Customer Engangement'.  I added 'Marketing' so that I can do a generic (email/web) marketing campaign.
 
<li>If you plan to use the CiviCase component to manage the common constituent "projects" or "workflows" and their associated timelines, then you'll need to create your own "Case Types". Look at the existing "Case Types" for reference.
 
<li>If you plan to use the CiviCase component to manage the common constituent "projects" or "workflows" and their associated timelines, then you'll need to create your own "Case Types". Look at the existing "Case Types" for reference.
<li>{{@todo}} review [http://wiki.civicrm.org/confluence/display/CRMDOC/Managing+Scheduled+Jobs docs] and setup cron to do things like geocoding.  '''Your mail campaigns will not send without cron''' Drupal cron is best managed with drush
+
<li>{{@todo}} review [http://wiki.civicrm.org/confluence/display/CRMDOC/Managing+Scheduled+Jobs docs] and setup cron to do things like geocoding.  '''Your mail campaigns will not send without cron'''
<li>Test and set your SPF record for your domain so that you can use Mailer, and review the docs for [http://book.civicrm.org/user/current/advanced-configuration/email-system-configuration/ email system configuration]  I was unable to get CiviCRM to use Google's smtp.gmail.com server, nor relay-smtp.gmail.com, even with an IP address whitelisted.  This is because Digital Ocean is still dropping all outbound SMTP traffic at their firewall (telnet doesn't even connect).  Somehow, if I smarthost it through [[Postfix]] it actually works.  I want to use Google for delivery because using <code>mail()</code> from an IP at Digital Ocean will result in mail being flagged as spam or silently dropped by several major providers (e.g. Yahoo, Microsoft).  Besides, if I'm '''paying''' for GAFYD, then I want to actually '''use''' it!  Google IS pretty well known for their email delivery capability!  Ultimately, Google is not a good choice for delivery.  Use a vendor. (See [[CiviCRM/CiviMail]] and [[Email Marketing]]
+
<li>Test and set your SPF record for your domain so that you can use Mailer, and review the docs for [http://book.civicrm.org/user/current/advanced-configuration/email-system-configuration/ email system configuration]
 
<li>Turn on logging in the Administration console, otherwise each record has a changelog, but there is no detail in the log!
 
<li>Turn on logging in the Administration console, otherwise each record has a changelog, but there is no detail in the log!
<li>Synchronize Users to Contacts
 
<li>Create a BOD group, and put each member into the group
 
 
</ol>
 
</ol>
  
Line 55: Line 44:
  
 
== Upgrading ==
 
== Upgrading ==
[[{{PAGENAMEE}}/upgrade]]
+
There is a long guide on [http://wiki.civicrm.org/confluence/display/CRMDOC/Upgrading+CiviCRM+for+Drupal+7 how to upgrade CiviCRM for drupal]  
  
== Debugging and the CV command-line ==
+
You should be familiar with all the steps described in the manual, and understand how the particular steps impact/affect your installation.  The short version below is known to work with our setup, and with 'routine' upgrades.
  
If you have your CiviCRM sending errors to the Drupal Watchdog Log (CiviCRM Administer > System Settings > [https://equality-tech.com/civicrm/admin/setting/debug?reset=1 Debugging and Error Handling]), you can simply navigate to it in the Drupal Admin (Reports > [https://equality-tech.com/admin/reports/dblog Recent log messages])
+
Here is the short version:
 +
# backup your database(s) <code>sudo ~/bin/backup.db.sh drupal</code> <code>sudo ~/bin/backup.db.sh civicrm</code> <ref>[[Mysqldump]]</ref>
 +
# download the code <code> wget https://download.civicrm.org/civicrm-4.6.5-drupal.tar.gz</code>
 +
# put site in maintenance mode<code>drush vset maintenance_mode 1</code> (Make sure you are also logged in as Admin)
 +
# move old code, and unpack new code <code>mv civicrm /tmp/ && tar xvzf civicrm-4.6.5-drupal.tar.gz</code>
 +
# run the upgrade script <code>/civicrm/upgrade?reset=1</code>
 +
# put site in operation mode<code>drush vset maintenance_mode 0</code>
 +
# toast
  
There are many tools you can use to debug your CiviCRM instance.
 
One of these is the <code>cv</code> tool [https://github.com/civicrm/cv available on Github].
 
<pre>
 
cv version v0.1.27
 
 
Usage:
 
  command [options] [arguments]
 
 
Options:
 
  -h, --help            Display this help message
 
  -q, --quiet          Do not output any message
 
  -V, --version        Display this application version
 
      --ansi            Force ANSI output
 
      --no-ansi        Disable ANSI output
 
  -n, --no-interaction  Do not ask any interactive question
 
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
 
 
Available commands:
 
  api                    Call an API
 
  cli                    Load interactive command line
 
  dis                    Disable an extension
 
  dl                      Download and enable an extension
 
  en                      Enable an extension
 
  ev                      Evaluate a snippet of PHP code
 
  flush                  Flush system caches
 
  help                    Displays help for a command
 
  list                    Lists commands
 
  path                    Look up the path to a file or directory
 
  scr                    Execute a PHP script
 
  url                    Compose a URL to a CiviCRM page
 
ang
 
  ang:html:list          List Angular HTML files
 
  ang:html:show          Show an Angular HTML file
 
  ang:module:list        List Angular modules
 
api
 
  api:batch              Call an API (batch mode)
 
debug
 
  debug:container        Dump the container configuration
 
  debug:event-dispatcher  Dump the list of event listeners
 
ext
 
  ext:disable            Disable an extension
 
  ext:download            Download and enable an extension
 
  ext:enable              Enable an extension
 
  ext:list                List extensions
 
  ext:uninstall          Uninstall an extension and purge its data
 
  ext:upgrade-db          Apply DB upgrades for any extensions
 
php
 
  php:boot                Generate PHP bootstrap code
 
  php:eval                Evaluate a snippet of PHP code
 
  php:script              Execute a PHP script
 
vars
 
  vars:fill              Generate a configuration file for any missing site data
 
  vars:show              Show the configuration of the local CiviCRM installation
 
</pre>
 
 
Also, don't forget to use [[Drush]] if you're on [[Drupal]]; the CiviCRM API; and the ConfigAndLog directory log.  The ConfigAndLog directory is in a path like <code>./sites/default/files/civicrm/ConfigAndLog</code>
 
  
 
== Profiles ==
 
== Profiles ==
Line 130: Line 70:
  
 
<blockquote>
 
<blockquote>
Custom tokens (based on custom data) can be added for organizations as well. These tokens will not be displayed in the list of available tokens, but can be added manually. The format is {contact.custom_12} where 12 is the ID of the custom data field. To find the custom data field ID, go Administer > Customize Data & Screens > Custom Fields and click �edit� on the field you want to use. Look at the URL. The last part of the URL will be an equal sign and a number (=12). The number (12 in this example) is the id of that custom field.
+
Custom tokens (based on custom data) can be added for organizations as well. These tokens will not be displayed in the list of available tokens, but can be added manually. The format is {contact.custom_12} where 12 is the ID of the custom data field. To find the custom data field ID, go Administer > Customize Data & Screens > Custom Fields and click on the field you want to use. Look at the URL. The last part of the URL will be an equal sign and a number (=12). The number (12 in this example) is the id of that custom field.
 
</blockquote>
 
</blockquote>
  
 
You can create your own tokens by implementing <code>hook_civicrm_tokens()</code> and  <code>hook_civicrm_tokenValues()</code>.  See  
 
You can create your own tokens by implementing <code>hook_civicrm_tokens()</code> and  <code>hook_civicrm_tokenValues()</code>.  See  
<source lang="php">
+
 
<?php
 
/**
 
* A convenience function so that we can map our custom fields and labels
 
* in one place, and share them between hooks without having to write to the db
 
* with variable_set()
 
*/
 
function getHaystack($stack='general') {
 
    $genKeys = array (
 
        "wUrl"      => "custom_40",
 
        "mainpage"  => "custom_41",
 
        "base"      => "custom_42",
 
        "sitename"  => "custom_43",
 
        "logo"      => "custom_44",
 
        "generator"  => "custom_45",
 
        "phpversion" => "custom_46",
 
    // "phpsapi",
 
        "dbtype"    => "custom_47",
 
        "dbversion"  => "custom_48",
 
    // "externalimages",
 
    // "langconversion",
 
    // "titleconversion",
 
    // "linkprefixcharset",
 
    // "linkprefix",
 
    // "linktrail",
 
    // "legaltitlechars",
 
    // "git-hash",
 
    // "git-branch",
 
    // "case",
 
    // "lang",
 
    // "fallback",
 
    // "fallback8bitEncoding",
 
        "writeapi"    => "custom_49",
 
        "timezone"    => "custom_50",
 
        "timeoffset"  => "custom_51",
 
        "articlepath" => "custom_52",
 
        "scriptpath"  => "custom_53",
 
    // "script",
 
    // "variantarticlepath",
 
        "server"      => "custom_54",
 
        "servername"  => "custom_55",
 
        "wikiid"      => "custom_56",
 
        "time"        => "custom_57",
 
        "maxuploadsize" => "custom_58",
 
    // "thumblimits",
 
    // "imagelimits",
 
        "favicon"    => "custom_59",
 
    );
 
    $statKeys = array(
 
        "wUrl"    => "custom_60",
 
        "pages"    => "custom_61",
 
        "articles" => "custom_62",
 
        "edits"    => "custom_63",
 
        "images"  => "custom_64",
 
        "users"    => "custom_65",
 
        "activeusers" => "custom_66",
 
        "admins"  => "custom_67",
 
        "jobs"    => "custom_68",
 
    );
 
    switch ($stack) {
 
        case 'general':
 
            $haystack = $genKeys;
 
            $addLabels = array (
 
                'recorded' => 'custom_69',
 
            );
 
            $haystack += $addLabels;
 
            break;
 
        case 'stats':
 
            $haystack = $statKeys;
 
            $addLabels = array (
 
                'recorded' => 'custom_70',
 
            );
 
            $haystack += $addLabels;
 
            break;
 
        default:
 
            die('no stack by that name');
 
    }
 
    return $haystack;
 
}
 
/**
 
* implementation of hook_civicrm_tokens()
 
* Much appreciation and thanks to Eileen McNaughton who helped me get the logic
 
* correct on this. cf.https://github.com/eileenmcnaughton/civicrm_views_token/blob/master/civicrm_views_token.module#L16
 
*
 
* In the end, we want to populate more $tokens in a format like
 
* $token['general.generator'] = 'Generator';
 
* $token['general.sitename'] = 'Sitename';
 
* where the $token key is the element that needs to be populated
 
* by hook_civicrm_tokenValues()
 
* The $token value here is used in the UI as a label for the token.
 
*/
 
function eqt_civicrm_tokens(&$tokens) {
 
   
 
    $labels = getHaystack('general');
 
    $tokens['general'] = array();
 
    foreach ($labels as $k => $v) {
 
        $tokens['general']["general.$k"] = "$k ($v)";
 
    }
 
 
 
    $labels = getHaystack('stats');
 
    $tokens['stats'] = array();
 
    foreach ($labels as $k => $v) {
 
        $tokens['stats']["stats.$k"] = "$k ($v)";
 
    }
 
}
 
/**
 
* implementation of hook_civicrm_tokenValues()
 
*/
 
function eqt_civicrm_tokenValues(&$values, $cids, $job = null, $tokens = array(), $context = null) {
 
    // for debugging
 
    //    watchdog('eqt',"eqt_civicrm_tokenValues(\$values, \$cids, \$job, \$tokens, \$context)<pre>\n\$values=" . var_export($values,1) . "\n\$cids=" . var_export($cids,1) . "\n\$job=" . $job ."\n\$tokens=" . var_export($tokens,1) . "\n\$context=" . $context . "\n</pre>");
 
   
 
    if ( array_key_exists('general', $tokens) ) {
 
        $haystack = getHaystack('general');
 
        foreach ($cids as $cid) {
 
            $params = array(
 
                'sequential' => 1,
 
                'entity_id' => $cid,
 
            );
 
            $result = civicrm_api3('CustomValue', 'get', $params);
 
            if (!$result['is_error']) {
 
                $customdata = ($result['values']);
 
                foreach ($customdata as $val) {
 
                    $needle = 'custom_' . $val['id'];
 
                    $label = array_search($needle, $haystack);
 
                    if (!$label) {
 
                        continue;
 
                    }
 
                    $label = 'general.' . $label;
 
                    $values[$cid][$label] = $val['latest'];
 
                }
 
            }
 
        }
 
    }
 
   
 
    if ( array_key_exists('stats', $tokens) ) {
 
        $haystack = getHaystack('stats');
 
        foreach ($cids as $cid) {
 
            $params = array(
 
                'sequential' => 1,
 
                'entity_id' => $cid,
 
            );
 
            $result = civicrm_api3('CustomValue', 'get', $params);
 
            if (!$result['is_error']) {
 
                $customdata = ($result['values']);
 
                foreach ($customdata as $val) {
 
                    $needle = 'custom_' . $val['id'];
 
                    $label = array_search($needle, $haystack);
 
                    if (!$label) {
 
                        continue;
 
                    }
 
                    $label = 'stats.' . $label;
 
                    $values[$cid][$label] = $val['latest'];
 
                }
 
            }
 
        }
 
    }
 
}
 
</source>
 
  
 
* [https://civicrm.stackexchange.com/questions/2558/tokens-for-custom-field-set-with-multiple-records?rq=1 Stack Exchange]
 
* [https://civicrm.stackexchange.com/questions/2558/tokens-for-custom-field-set-with-multiple-records?rq=1 Stack Exchange]
 
* https://civicrm.org/blogs/colemanw/create-your-own-tokens-fun-and-profit Coleman's writeup
 
* https://civicrm.org/blogs/colemanw/create-your-own-tokens-fun-and-profit Coleman's writeup
  
* http://www.gingerfeet.net/civicrm-userguide-customdatafields <ref>[http://www.gingerfeet.com/home GingerFeet] is a Drupal and CiviCRM consultant + hosting provider like {{CompanyName}}</ref> has an overview that is pulled from the manual, however it may not reflect the most current version - especially since they sell CiviCRM as a service.
+
* http://www.gingerfeet.net/civicrm-userguide-customdatafields <ref>[http://www.gingerfeet.com/home GingerFeet] is a Drupal and CiviCRM consultant + hosting provider like [https://eQuality-Tech.com eQuality Technology]</ref> has an overview that is pulled from the manual, however it may not reflect the most current version - especially since they sell CiviCRM as a service.
 
* http://book.civicrm.org/user/current/organising-your-data/custom-fields/ the manual itself
 
* http://book.civicrm.org/user/current/organising-your-data/custom-fields/ the manual itself
  
Line 314: Line 96:
 
== Extensions ==
 
== Extensions ==
 
* See [https://civicrm.org/extensions/drupal listing of CiviCRM extensions for Drupal]
 
* See [https://civicrm.org/extensions/drupal listing of CiviCRM extensions for Drupal]
* [https://docs.civicrm.org/sysadmin/en/latest/customize/extensions/#installing-a-new-extension  Installing extensions]
+
* [http://wiki.civicrm.org/confluence/display/CRMDOC/Extensions Extensions documentation in the wiki]
There are multiple ways to use the <code>cv</code> command to download and install extensions:
+
* [http://wiki.civicrm.org/confluence/display/CRMDOC/Create+a+Custom-Search+Extension Create a Custom Search extension]
;Download a published extension from the directory (long name).
 
:<code>cv dl org.example.foobar</code>
 
;Download a published extension from the directory (short name).
 
:<code>cv dl foobar</code>
 
;Download an unpublished extension (long name and zip URL)
 
:<code>cv dl org.example.foobar@http://example.org/files/foobar-1.2.zip</code>
 
;Download a pre-release (alpha/beta) from the directory.
 
:<code>cv dl --dev foobar</code>
 
 
 
 
 
  
 
== Developing Extensions ==
 
== Developing Extensions ==
Line 346: Line 118:
  
 
== Database ==
 
== Database ==
The database code in CiviCRM is divided into two logical sections: the DAO and the BAO.  The BAO holds the "business logic" for objects and extends the DAO.  The DAO is concerned with data to/from the database backend and it's definition (object to relational database mapping, aka "[[wp:Object-relational mapping|ORM]]").  Both are an extension of the [https://pear.php.net/manual/en/package.database.db-dataobject.php PEAR DB DataObject] class.  As of August 2015, this is true and [https://civicrm.org/node/95 this blog post from 2006] gives some more (early) background.
+
The database code in CiviCRM is divided into two logical sections: the DAO and the BAO.  The BAO holds the "business logic" for objects and extends the DAO.  The DAO is concerned with data to/from the database backend and it's definition (object to relational database mapping, aka "<abbr title="Object-relational Mapping">[[wp:Object-relational mapping|ORM]]</abbr>").  Both are an extension of the [https://pear.php.net/manual/en/package.database.db-dataobject.php PEAR DB DataObject] class.  As of August 2015, this is true and [https://civicrm.org/node/95 this blog post from 2006] gives some more (early) background.
 
<source lang="php">
 
<source lang="php">
 
// from CRM/core/DAO.php
 
// from CRM/core/DAO.php

Revision as of 14:28, 23 August 2015