Difference between revisions of "Testing"

From Freephile Wiki
Jump to navigation Jump to search
Line 2: Line 2:
  
 
Software development is writing code. Testing makes sure the code actually works, so in a nutshell: '''Testing is software development''' :-)
 
Software development is writing code. Testing makes sure the code actually works, so in a nutshell: '''Testing is software development''' :-)
 
+
== Phan ==
==Phan==
+
see [[Phan]]
Phan is a '''[[wp:Static program analysis|static analyzer]]''' for PHP.  It will help you write better PHP code. Please note that your source code  To use it, you'll need to install the [https://github.com/nikic/php-ast Abstract Syntax Tree] PHP extension.  See the [https://github.com/phan/phan/wiki/Tutorial-for-Analyzing-a-Large-Sloppy-Code-Base tutorial for analyzing a large sloppy codebase]
 
 
 
[https://github.com/phan/phan Phan project on GitHub]
 
 
 
 
 
The [[MediaWiki]] project uses Phan in its [[Continuous Integration]].
 
 
 
===Links===
 
 
 
#[https://github.com/wikimedia/mediawiki-tools-phan mediawiki-tools-phan] on github
 
#[https://github.com/phan/phan phan/phan] on github
 
#[https://github.com/nikic/php-ast nikic/php-ast] on github
 
#Phan [https://github.com/phan/phan/wiki/Getting-Started Getting-Started]
 
#Phan [https://github.com/phan/phan/blob/v5/internal/CLI-HELP.md CLI-HELP.md]
 
 
 
===Static Analysis of MediaWiki===
 
For some current analysis see https://doc.wikimedia.org/mediawiki-core/master/phpmetrics/complexity.html
 
 
 
#[https://www.mediawiki.org/wiki/Continuous_integration/Entry_points Continuous_integration/Entry_points]
 
#[https://www.mediawiki.org/wiki/Continuous_integration/Phan Continuous_integration/Phan]
 
#[[mediawikiwiki:Continuous_integration/Tutorials/Add_phan_to_a_MediaWiki_extension|Continuous_integration/Tutorials/Add_phan_to_a_MediaWiki_extension]]
 
#[https://www.mediawiki.org/wiki/Best_practices_for_extensions#File_structure Best_practices_for_extensions#File_structure]
 
 
 
The on-wiki documentation (at MediaWiki.org) and even the upstream projects do not exactly provide a usable guide for MediaWiki implementors to use Phan on an extensive codebase ('''ie. "my whole wiki")'''. Instead, the documentation describes how '''MediaWiki extension''' developers can use Phan by incorporating their code into the larger configuration of MediaWiki Continuous Integration. If you download MediaWiki and composer update (to get dev dependencies) and also install PHP-AST, then you should be able to run <code>composer phan</code> or <code>./vendor/bin/phan -p .</code> But, that analyzes the entire MediaWiki codebase (using a lot of RAM and time), and does '''not''' analyze random extensions (found on client's wiki instance) unless those extensions already have their own <code>./phan/config.php</code>
 
 
 
The traditional form of that<ref>https://codesearch.wmcloud.org/things/?q=exclude_analysis_directory_list&files=.phan%2Fconfig.php&excludeFiles=&repos=</ref>, for an extension named "MyExtension", which has dependencies on the Echo and SocialProfile extension, is<syntaxhighlight lang="php">
 
$cfg = require __DIR__ . '/../vendor/mediawiki/mediawiki-phan-config/src/config.php';
 
 
 
$cfg['directory_list'] = array_merge(
 
$cfg['directory_list'],
 
[
 
'../../extensions/Echo',
 
'../../extensions/SocialProfile',
 
]
 
);
 
$cfg['exclude_analysis_directory_list'] = array_merge(
 
$cfg['exclude_analysis_directory_list'],
 
[
 
'../../extensions/Echo',
 
'../../extensions/SocialProfile',
 
]
 
);
 
 
 
return $cfg;
 
</syntaxhighlight>The reason the normal pattern is to include directories and exclude those same directories is that you include 'dependencies' for class definitions etc. (symbol discovery), but do not warn about issues outside the limits of your own MyExtension extension.
 
 
 
{{Ambox|
 
|text=If your Extension '''does''' have dependencies on other extensions, '''and''' you are hosting your code in gerrit for automated CI checks and fixes (you should), you must create an entry in the so-called [https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/refs/heads/master/zuul/parameter_functions.py#558 parameter-functions.py] to declare those dependencies.}}
 
 
 
====MediaWiki Phan Config====
 
The bigger (unanswered) question is '''what does mediawiki-phan-config do?''' What is the compiled and complete configuration that you are "running" against your codebase?
 
 
 
<br />
 
 
 
{{Collapsible
 
|visible_text=Show the full configuration for MediaWiki phan
 
|collapsed_content=
 
 
 
In <code>.phan/config.php</code> I simply added <syntaxhighlight lang="php">print_r( $cfg ); exit();</syntaxhighlight> just before the 'return' at the end of the file. Then I ran <code>composer phan</code> to invoke it from my shell and display the object variable.
 
<syntaxhighlight lang="php" line="1">
 
Array
 
(
 
    [backward_compatibility_checks] =>
 
    [parent_constructor_required] => Array
 
        (
 
        )
 
 
 
    [quick_mode] =>
 
    [analyze_signature_compatibility] => 1
 
    [ignore_undeclared_variables_in_global_scope] => 1
 
    [read_type_annotations] => 1
 
    [disable_suppression] =>
 
    [dump_ast] =>
 
    [dump_signatures_file] =>
 
    [processes] => 1
 
    [whitelist_issue_types] => Array
 
        (
 
        )
 
 
 
    [markdown_issue_messages] =>
 
    [generic_types_enabled] => 1
 
    [plugins] => Array
 
        (
 
            [0] => PregRegexCheckerPlugin
 
            [1] => UnusedSuppressionPlugin
 
            [2] => DuplicateExpressionPlugin
 
            [3] => LoopVariableReusePlugin
 
            [4] => RedundantAssignmentPlugin
 
            [5] => UnreachableCodePlugin
 
            [6] => SimplifyExpressionPlugin
 
            [7] => DuplicateArrayKeyPlugin
 
            [8] => UseReturnValuePlugin
 
            [9] => AddNeverReturnTypePlugin
 
            [10] => /opt/htdocs/mediawiki/vendor/mediawiki/mediawiki-phan-config/src/../../phan-taint-check-plugin/MediaWikiSecurityCheckPlugin.php
 
        )
 
 
 
    [plugin_config] => Array
 
        (
 
        )
 
 
 
    [file_list] => Array
 
        (
 
            [0] => .phan/stubs/password.php
 
            [1] => .phan/stubs/Socket.php
 
            [2] => .phan/stubs/WeakMap.php
 
            [3] => includes/Defines.php
 
            [4] => tests/phpunit/MediaWikiIntegrationTestCase.php
 
            [5] => tests/phpunit/includes/TestUser.php
 
        )
 
 
 
    [exclude_file_list] => Array
 
        (
 
            [0] => vendor/squizlabs/php_codesniffer/src/Standards/PSR2/Tests/Methods/MethodDeclarationUnitTest.inc
 
            [1] => vendor/php-parallel-lint/php-parallel-lint/src/polyfill.php
 
        )
 
 
 
    [exclude_analysis_directory_list] => Array
 
        (
 
            [0] => vendor/
 
            [1] => .phan/
 
            [2] => tests/phpunit/
 
            [3] => includes/config-vars.php
 
            [4] => includes/composer/
 
            [5] => maintenance/language/
 
            [6] => includes/libs/jsminplus.php
 
            [7] => includes/libs/objectcache/utils/MemcachedClient.php
 
            [8] => includes/PHPVersionCheck.php
 
        )
 
 
 
    [exclude_file_regex] => @vendor/((composer/installers|php-parallel-lint/php-console-color|php-parallel-lint/php-console-highlighter|php-parallel-lint/php-parallel-lint|mediawiki/mediawiki-codesniffer|microsoft/tolerant-php-parser|phan/phan|phpunit/php-code-coverage|squizlabs/php_codesniffer|[^/]+/[^/]+/\.phan)|.*/[Tt]ests?)/@
 
    [minimum_severity] => 0
 
    [allow_missing_properties] =>
 
    [null_casts_as_any_type] =>
 
    [scalar_implicit_cast] =>
 
    [dead_code_detection] =>
 
    [dead_code_detection_prefer_false_negative] => 1
 
    [progress_bar] =>
 
    [enable_class_alias_support] =>
 
    [redundant_condition_detection] => 1
 
    [minimum_target_php_version] => 7.4.3
 
    [target_php_version] => 8.1
 
    [directory_list] => Array
 
        (
 
            [0] => includes/
 
            [1] => languages/
 
            [2] => maintenance/
 
            [3] => mw-config/
 
            [4] => resources/
 
            [5] => vendor/
 
            [6] => tests/common/
 
            [7] => tests/parser/
 
            [8] => tests/phpunit/mocks/
 
        )
 
 
 
    [suppress_issue_types] => Array
 
        (
 
            [0] => PhanDeprecatedFunction
 
            [1] => PhanDeprecatedClass
 
            [2] => PhanDeprecatedClassConstant
 
            [3] => PhanDeprecatedFunctionInternal
 
            [4] => PhanDeprecatedInterface
 
            [5] => PhanDeprecatedProperty
 
            [6] => PhanDeprecatedTrait
 
            [7] => PhanUnreferencedUseNormal
 
            [8] => PhanUnreferencedUseFunction
 
            [9] => PhanUnreferencedUseConstant
 
            [10] => PhanDuplicateUseNormal
 
            [11] => PhanDuplicateUseFunction
 
            [12] => PhanDuplicateUseConstant
 
            [13] => PhanUseNormalNoEffect
 
            [14] => PhanUseNormalNamespacedNoEffect
 
            [15] => PhanUseFunctionNoEffect
 
            [16] => PhanUseConstantNoEffect
 
            [17] => PhanDeprecatedCaseInsensitiveDefine
 
            [18] => PhanAccessClassConstantInternal
 
            [19] => PhanAccessClassInternal
 
            [20] => PhanAccessConstantInternal
 
            [21] => PhanAccessMethodInternal
 
            [22] => PhanAccessPropertyInternal
 
            [23] => PhanParamNameIndicatingUnused
 
            [24] => PhanParamNameIndicatingUnusedInClosure
 
            [25] => PhanProvidingUnusedParameter
 
            [26] => PhanPluginMixedKeyNoKey
 
            [27] => SecurityCheck-LikelyFalsePositive
 
            [28] => SecurityCheck-PHPSerializeInjection
 
            [29] => PhanPluginDuplicateExpressionAssignmentOperation
 
            [30] => PhanPluginDuplicateExpressionAssignmentOperation
 
        )
 
 
 
    [globals_type_map] => Array
 
        (
 
            [wgContLang] => \Language
 
            [wgParser] => \Parser
 
            [wgTitle] => \Title
 
            [wgMemc] => \BagOStuff
 
            [wgUser] => \User
 
            [wgConf] => \SiteConfiguration
 
            [wgLang] => \Language
 
            [wgOut] => OutputPage
 
            [wgRequest] => \WebRequest
 
            [IP] => string
 
            [wgGalleryOptions] => array
 
            [wgDummyLanguageCodes] => string[]
 
            [wgNamespaceProtection] => array<int,string|string[]>
 
            [wgNamespaceAliases] => array<string,int>
 
            [wgLockManagers] => array[]
 
            [wgForeignFileRepos] => array[]
 
            [wgDefaultUserOptions] => array
 
            [wgSkipSkins] => string[]
 
            [wgLogTypes] => string[]
 
            [wgLogNames] => array<string,string>
 
            [wgLogHeaders] => array<string,string>
 
            [wgLogActionsHandlers] => array<string,class-string>
 
            [wgPasswordPolicy] => array<string,array<string,string|array>>
 
            [wgVirtualRestConfig] => array<string,array>
 
            [wgWANObjectCaches] => array[]
 
            [wgLocalInterwikis] => string[]
 
            [wgDebugLogGroups] => string|false|array{destination:string,sample?:int,level:int}
 
            [wgCookiePrefix] => string|false
 
            [wgExtraNamespaces] => string[]
 
        )
 
 
 
    [analyzed_file_extensions] => Array
 
        (
 
            [0] => php
 
            [1] => inc
 
        )
 
 
 
    [autoload_internal_extension_signatures] => Array
 
        (
 
            [dom] => .phan/internal_stubs/dom.phan_php
 
            [excimer] => .phan/internal_stubs/excimer.php
 
            [imagick] => .phan/internal_stubs/imagick.phan_php
 
            [memcached] => .phan/internal_stubs/memcached.phan_php
 
            [oci8] => .phan/internal_stubs/oci8.phan_php
 
            [pcntl] => .phan/internal_stubs/pcntl.phan_php
 
            [pgsql] => .phan/internal_stubs/pgsql.phan_php
 
            [redis] => .phan/internal_stubs/redis.phan_php
 
            [sockets] => .phan/internal_stubs/sockets.phan_php
 
            [sqlsrv] => .phan/internal_stubs/sqlsrv.phan_php
 
            [tideways] => .phan/internal_stubs/tideways.phan_php
 
            [wikidiff2] => .phan/internal_stubs/wikidiff.php
 
        )
 
 
 
)
 
 
 
</syntaxhighlight>
 
 
 
}}
 
 
 
 
 
MediaWiki Phan Config has a [https://github.com/wikimedia/mediawiki-tools-phan/blob/master/src/ConfigBuilder.php#L8 ConfigBuilder.php] class that is used to build up the project's phan config. The goto file to see how it's configured is '''[https://github.com/wikimedia/mediawiki-tools-phan/blob/master/src/config.php config.php]'''
 
 
 
Phan has a ton of plugins - but their usage in the MediaWiki configuration is not mentioned or described anywhere (that I could find). I found a list in 'base-config-functions' [https://github.com/wikimedia/mediawiki-tools-phan/blob/master/src/base-config-functions.php#L68 mediawiki-tools-phan/blob/master/src/base-config-functions.php#L68], but why you need to read the entire source code to "reverse engineer" what is happening? It's not fun.
 
 
 
===Phan Plugins===
 
https://github.com/phan/phan/tree/v5/.phan/plugins#2-general-use-plugins
 
 
 
The [https://github.com/phan/phan/wiki/Writing-Plugins-for-Phan writing plugins guide] also describes how they work.
 
 
 
===Check code for compatibility with a particular PHP version===
 
One major reason I wanted to use phan was to upgrade a whole codebase, and check its compatibility with an upgraded PHP (moving from 7.4 to 8.1). There is at least one caveat to doing this: You should be running php 8.1 in order to check code for compatibility with 8.1 You can use phan with the config setting of  <code>target_php_version</code><ref>https://github.com/phan/phan/wiki/Phan-Config-Settings#analysis-of-a-php-version</ref>
 
 
 
===Phan with VSCode===
 
https://github.com/phan/phan/wiki/Editor-Support supposed to just work (with the plugin). But after I installed the plugin, it was not producing any hint of working or any errors.
 
 
 
===Phan checks===
 
There are over 1,000 things that phan tests its own code against
 
 
 
https://github.com/phan/phan/tree/v5/tests
 
 
 
https://github.com/phan/phan/tree/v5/tests/files/src
 
<br />
 
  
 
==Other Static Analysis tools for PHP==
 
==Other Static Analysis tools for PHP==

Revision as of 17:47, 18 February 2024

Testing is software development.

Software development is writing code. Testing makes sure the code actually works, so in a nutshell: Testing is software development :-)

Phan[edit | edit source]

see Phan

Other Static Analysis tools for PHP[edit | edit source]

PHPStan[edit | edit source]

https://phpstan.org/

PHPStan seems more polished (perhaps because it's commercial and has a 'pro' version that adds a GUI) whereas phan is the original PHP static analysis tool Rasmus Ledorf uses.

Adding a configuration file for your MediaWiki extension is straightforward and would look like this:

parameters:
	level: 1
	paths:
		- src
		- tests
	scanDirectories:
		- ../../includes
		- ../../tests/phpunit
		- ../../vendor


The level can be 0 - 9

The paths are the directories of your code.

The scanDirectories are additional paths used to discover symbols, but not analyze for errors.

For more advanced usage, see the example of Professional Wiki's Maps extension which illustrates configuration file includes of a 'baseline'; error message suppression; and directory exclusions of problem code.

Psalm[edit | edit source]

https://psalm.dev/