CiviCRM/debugging

on line 524 of civicrm/api/V3/Mailing.php, in civicrm_api3_mailing_preview($params) there is a call to CRM_Mailing_BAO_Mailing::tokenReplace($mailing);

The $mailing is an array with 9 elements, all 'private'

_tableName is "civicrm_mailing"
_fields
_fieldKeys is array[39]

The $body_html is the content of the message and includes token literals

Stepping into that function

loadClass() ends up loading $file CRM/Utils/Token.php (when $class="CRM_Utils_Token")

getTokens($string) is called and returns things like

$tokens['general'][] = "wUrl"

gets passed to _getTokens($prop) and $newTokens is the same as $tokens

eventually tokenReplace() gets called

$details gets populated at line 536 of civicrm/api/V3/Mailing.php and that array has the data, but the keys are like 'custom_40', not general.wUrl

$details[0][2] = array(21) 
[contact_id] "2"
[custom_40] "https://freephile.org/wiki/Main_Page"
[custom_45] "MediaWiki 1.25beta"
[stats.wUrl] "https://freephile.org/wiki/Main_Page"
[stats.articles]
[stats.edits]

$details[1][40][attributes]= array[3] 
'label', 
'data_type', 
'html_type'
$details[1][45][attributes]

echo self::output($result); is called from REST.php

Second pass through

So I ran the debugger again. This time with another "Custom Data Fieldset" and element (Called "Dummy -> Dufus"). The only difference was that this field was 'singular' rather than multi-value.

When I click "Preview as HTML", the value is correctly displayed for the first contact in my "Recipient list". The token is {contact.custom_71}

I used a breakpoint in civicrm/api/V3/Mailing.php at line 536 where $details gets populated from a call to CRM_Utils_Token::getTokenDetails($mailingParams, $returnProperties, TRUE, TRUE, NULL, $mailing->getFlattenedTokens())

At that point, the variable $tokens is an array[3] with keys html, text, subject

$tokens['html'] is an array[4] with keys contact, general, domain, action
$tokens['html']['contact'] is an array[3] with values 'first_name', 'custom_40' and 'custom_71'
$tokens['html']['general'] is an array[2] with values 'wUrl', 'custom_40'
$tokens['html']['domain'] is an array[1] with values 'address'
$tokens['html']['action'] is an array[1] with values 'optOutUrl'

At this point, $body_html contains the un-evaluated html (meaning the tokens are still present in their literal form).

Interestingly, the $_query is data_select * from $_table civicrm_mailing

$returnProperties is an array[7] with hash, preferred_mail_format, contact_id, display_name, first_name, custom_40, custom_71 all set to (int) 1

Some related constants?

FIELD_BEHAVIOR_NONE 1
FIELD_BEHAVIOR_DEFAULT 2
FIELD_BEHAVIOR_CUSTOM 4
FIELD_LOAD_CURRENT "FIELD_LOAD_CURRENT"
FIELD_LOAD_REVISION "FIELD_LOAD_REVISION"
FIELD_TYPE_NONE "undefined"

The first thing that happens when you step into CRM_Utils_Token::getTokenDetails() is that $mailing->getFlattenedTokens() is called.

There is a $this->tokens, but not a $this->flattenedTokens, so $this->getTokens() on line 873 is a 'no op' (tokens are already defined).

Then we get into a bunch of class autoloader stuff and CRM_Core_Form results in CRM/Core/Form.php being loaded after checking various caches for both class code and interface code. CRM_Core_Form extends HTML_QuickForm_Page

There is a check at line 1218 of Token.php for empty($returnProperties) that would then return all properties, but we do already have defined $returnProperties.

getKeyID is called and for $key "custom_40", it returns "40" since $all is bool 0
getKeyID is called and for $key "custom_71", it returns "71" since $all is bool 0

Those values populate $cfID and in turn $custom[], so that now $custom is array("40", "71")

We then go through the autoloader mechanism for "CRM_Contact_BAO_Query"

exportableFields() gets called, and the 5th argument in that signature is "$withMultiRecord=FALSE" however I don't think that the callee specifies a value, and so the variable is set to bool(0)

$cacheKeyString ends up being "exportableFields All_1_0_1_2"

and $argString is "exportableFields All_1_0_1_2"

Cache::getItem() is called and Array::value() is called on list which looks small at array(15) but elements like [CRM_PC_CRM_Core_DAO_StateProvince_1_id_name_is_active] is array(3907)

Then we go through the DataObject class methods to create a dsn and connection

some of the queries issued:

SELECT *,  config_backend, locales, locale_custom_strings  FROM civicrm_domain WHERE ( civicrm_domain.id = 1 );
SELECT subtype.*, parent.name as parent, parent.label as parent_label FROM   civicrm_contact_type subtype INNER JOIN civicrm_contact_type parent ON subtype.parent_id = parent.id WHERE  subtype.name IS NOT NULL AND subtype.parent_id IS NOT NULL   AND subtype.is_active = 1 AND parent.is_active = 1 ORDER BY parent.id;
SELECT * FROM civicrm_mailing_group WHERE (  civicrm_mailing_group.id = 511 );
SELECT * FROM civicrm_mailing WHERE ( civicrm_mailing.id = 17 );
SELECT * FROM civicrm_domain WHERE ( civicrm_domain.id = 1 );

As CRM_Core_BAO_Cache::getItem() is called, $data gets set, and cached, by unserializing $dao->data. $data is now array[117]

and includes all the 'custom' fields such as

[custom_40] array[10] 'name', 'title', 'headerPattern', 'import', 'custom_filed_id' => (int) 40, 'text_length', 'data_type'=>"String", 'html_type'=>"Text", 'is_search_range'=>"0"

[custom_71]

That then gets stuffed into $fields array[117]

On line 1605 of CRM/Contact/BAO/Contact.php, $fields gets set as 'exportable'

self::$_exportableFields[$cacheKeyString] = $fields;

but a line later, because $status is false, $fields gets set to the stored value of $_exportableFields[$cacheKeyString] BUT inspecting that value in NetBeans before the assignment on line 1609,

self::$_exportableFields[$cacheKeyString]; says it's type array[117] but Netbeans only shows it as having 30 elements ????

But, that must be some quirk of NetBeans because after the assignment, $fields is indeed an array with 117 elements

later we get to __construct() in CRM/Contact/BAO/Query.php but $fields is null, so on line 445

$this->_fields = CRM_Contact_BAO_Contact::exportableFields('All', FALSE, TRUE, TRUE);

NOTICE THE 5th parameter ($withMultiRecord) is missing, so the default bool(0) is used. I overrode it and set it to 1

Later, $this->_fields = array_merge($this->_fields, $fields); array[117], array[122] = array[239]

Then on line 455, it says // add any fields provided by hook implementers

but $extFields is array[0] and somehow $this->_fields jumped up to array[255]

The function name / pattern that they are looking for is eqt_civicrm_queryObjects()

Somewhere along the line, around here, Symfony\Component\EventDispatcher::dispatch is called and the $event is an Civi\API\Event\RespondEvent object with several properties including the apiRequest with is an array[9] and apiRequest[fields] is array[123] and apiRequests[fields][custom_40] is an array[21] so this is where the API request and response is happening and dispatch gets called.

Later when runHooks() of Hook.php is called we're looking for a function named "eqt_civicrm_apiWrappers" for the eqt module

At line 1242 of CRM/Utils/Token.php $contactDetails = &$details[0]; and thus

$contactDetails array[1]
$contactDetails[2] array[12]
$contactDetais[2][custom_40] = "https://freephile.org/wiki/Main_Page"
$contactDetais[2][custom_71] = "Not, Never, Well Maybe"

Then runHooks() in CRM/Utils/Hook.php is called; and it's called with

$civiModules, $fnSuffix, $numParams, &$arg1, &$arg2, &$arg3, &$arg4, &$arg5, &$arg6

where

$civiModules is array[98]
$fnSuffix = civicrm_tokenValues
$numParams = 5
$arg1 is the $contactDetails
$arg2 is array('contact_id' => 2)
$arg3 is null
$arg4 is array[4]
$arg5 is null
$arg6 is null

Results

Unfortunately,

Hi {contact.first_name}
wUrl data is stored as custom 40
but the token in the UI is {general.wUrl}
custom_40 {custom_40}
contact.custom_40 {contact.custom_40}
general.custom_40 {general.custom_40}
general.general.custom_40 {general.general.custom_40}

This is the dufus {contact.custom_71}

produces

Hi Greg
wUrl data is stored as custom 40
but the token in the UI is 
custom_40 {custom_40}
contact.custom_40 {contact.custom_40}
general.custom_40 
general.general.custom_40 {general.general.custom_40}

This is the dufus Not, Never, Well Maybe

HyperText Markup Language = structured text that uses logical links (hyperlinks) between nodes containing text.

Application Programming Interface