'''
Name: avi_tpr.py

Overview: Provides methods to update the AVI TRP field

Description:
    This class provides the properties and methods to calculate and update
    AVI TPR values. These calculations are based on the AVI 2.1.1 March 2005
    specifications.

    The calculations are performed using a set of formulas that will determine
    TPR based on species groups. These formulas are based on the site index
    curves for these groups. The species groups are:

    Deciduous - Aw, Bw, Pb, A

    White spruce/Fir - Sw, Se, Fd, Fb, Fa

    Pine - P, Pl, Pj, Pa, Pf

    Black spruce/Larch - Sb, Lt, La, Lw
        Note that the specification has placed Larch in seperate group but
        since the site index formula for this group is identical to Black
        spruce it was assigned to this species group.

    The original formulas were implemented in VBA through the IAS application
    and these formulas were copied from this code and modified to python.
    Some changes were made to modifiy the upper and lower range of values used
    to determing the AVI TPR call from the site index calculation. These changes
    were made after consultation with Darren Aitkin to ensure these changes
    were acceptable.

    The site index formulas use the height and breast height age of the
    leading species in an AVI layer (overstorey/understorey). Breast height
    age is calculated by taking the layers origin and subtracting it from
    the photo year of the imagery used to classify the AVI. A age adjustment
    is then applied based on species group to obtain the breast height age.
    For instance, the adjustment for pine species is 10 years so 10 would
    be subtracted from the age calculated by subtracting origin from photo year.
    The adjustment values for the various species groups is hardcoded as a
    property of class.

    For most layers the height is directly taken from the AVI height field. The
    only possible exception to this is in the case of complex stands.
    The height of a complex stand represents the median height of the stand.
    How this median varies is indicated by a structure modifier that indicates
    the range of height. For instance, if the complex height was 10 and the
    structure modifier is 4 then this means the height of trees varies between
    8 and 12 meters. The application enforces the mid-point height for TPR calculations
    a complex stand.
    
    The origin determined for a stand depends on whether the origin is a generic
    decade call or an absolute origin determined to the year. With a decade the
    age of the layer can span 10 years. In these cases an adjustment is performed
    adding on 5 years so age repesents middle of decade. To determine if the origin
    is a decade origin the end of the origin is examined and if 0 then assume is
    decade and make adjustment. Will not always work but is sufficient for the vast
    majority of stands.
    
    During the assignment of TPR certain business rules are also enforced. For
    instance, when a stand is multilayered and the leading species in each layer
    is the same then the TPR assigned to the second layer is taken from the first
    layer.

    Where the layer is non-forest vegetated then the existing data field should
    indicate that the TPR is interpreted. The exception to this rule is where there
    is a multi-layered stand and the other layer is forested. In this case the TPR
    is derived from the forested layer and assigned to the non-forest vegetated layer.

    In cases where a TPR cannot be calculated because of height or is non-forest vegetated
    the original TPR for the layer is used if there is one. This is to ensure that TPRs
    that were originally assigned will not be deleted in case they were interpreted.

    In cases where the TPR cannot be calculated because of height or other issues
    then the Existing Data field will be calculated to I if is does not have a
    value. This should flag fields so an error message is generated through the attribute
    QC indicating to interpreter where an interpreted TPR is required.
    
Notes:

Author: Doug Crane
        Apr, 2011

Modifications:
    May 9, 2011 When a TPR cannot be calculated because height is less than
    2 m then the DATA field is calculated to I to flag stand as requiring an
    interpreted TRP. The same applies to NFL layers. (DC)

    Upgraded to 10.1 Sept 2012 (DC)

    Modified to perform TPR updates for layers greater than 5 meters. Also
    incorporated the validation routines. Oct 2016 DC

'''

import pdb
import os
import sys
import math
import datetime

import arcpy

from srd.srd_exception import *
import srd.srd_logging as srd_logging
import srd.srd_misc as srd_misc
import srd.srd_recordset as srd_recordset
import srd.srd_featureclass_info as srd_featureclass_info
import srd.srd_audit_log as srd_audit_log

from avi_record import *

__author__ = 'Doug Crane'
__version__ = '1.0'

__all__ = ['AVI_TPR']

VERBOSE=0
# ----------------------------------------------------------
class AVI_TPR(object):
    '''

    '''
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def version(self):
        '''
        Current version of software.
        '''
        return 'Sept 2, 2020'
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def __init__(self):
        '''
        Parameters:
        '''
        
        self._logChn = srd_logging.SRD_Log()
        
        # Breast height adjustments to origins based on species.
        self.AAdjustment = 6
        self.PAdjustment = 10
        self.SbAdjustment = 20
        self.SwAdjustment = 15

        # The decade adjust determines how to calculate the origin
        # when determining age. If a generic decade is used then will
        # take mid-point of decade as origin year since the origin can
        # span 10 years. We assume it is a decade origin if the origin
        # ends in 0. If does not then assume it is an absolute origin
        # and do not adjust it.
        self.adjustDecadeYrs = 5
        
        # Codes used to determine the species type to use
        # when calculating TRP values.
        self.whiteSpruceFirType = 'SW_FIR'
        self.decidType = 'DECID'
        self.pineType = 'PINE'
        self.blackSpruceLarchType = 'SB_LT'

        self.changeDict = {}
        
        if VERBOSE:
            self._logChn.logMsg('\n***\n*** VERBOSE IS ON\n***')

    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def getRequiredFields(self, fcPath):
        '''
        Returns the set of required fields. This include the field in the
        feature class representing the Object ID. The Object ID field  will
        always be the last field name in our list
        '''

        reqFieldList = ['tpr', 'utpr', 'sp1', 'usp1', 'data', 'udata', 'photo_yr',
                        'struc', 'ustruc', 'height', 'uheight', 'nfl', 'unfl', 'nat_non',
                        'unat_non', 'anth_veg', 'uanth_veg', 'origin', 'uorigin',
                        'data_yr', 'udata_yr']

        # Also required the OID of the feature class
        descObj = arcpy.Describe(fcPath)
        if descObj.hasOID:
            oidFieldName = descObj.OIDFieldName.lower()
            reqFieldList.append(oidFieldName)
        else:
            reqFieldList.append('no_oid')
            
        return reqFieldList
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def requiredFieldsPresent(self, fcPath):
        '''
        Validate that the required fields for TPR calculation are present

        Parameters:
            fcPath - path to table or feature class being validated.

        Returns:
            True - all required fields foudn
            False - required field found missing
        '''

        self._logChn.logMsg('\n*** Checking for required fields...')

        errList = []
        
        descObj = arcpy.Describe(fcPath)
        if descObj.hasOID:
            pass
        else:
            errList.append('Required ObjectID field not found')

        reqFieldList = self.getRequiredFields(fcPath)

        fcFieldNameList = [field.name.lower() for field in arcpy.ListFields(fcPath)]

        for fieldName in reqFieldList:
            if fieldName != 'no_oid' and not fieldName in fcFieldNameList:
                errList.append('Required field: %s missing' % fieldName)

        # Print error messages and return False to indicate error
        if errList:
            self._logChn.logWarning('\n***\n*** Required field(s) missing, cannot process:\n***')
            for err in errList:
                self._logChn.logWarning('%s\n' % err)
            return False
        else:
            self._logChn.logMsg('Required fields found')
            return True
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def calcTPR(self, speciesType, height, brHeightAge):
        '''
        Calculates the TPR value. Runs the formula to determine
        where stand is on site index curve based on the height
        and breast height age of leading species then determines
        TPR based on where it falls within range of values determined
        for each TPR classification code.

        Formulas taken from AVI 2.1.1 March 2005 specifications. Note
        that the formula for SB and LT are same so the LT types are
        processed with the SB formula.

        Parameters:
            speciesType - Species class used to determine method for
                          calculating TPR.

            height - layer height

            brHeightAge - age at breast height.

        Returns
            a tuple containing the calculated TPR value (U,F,M,G) and the
            TPR calculation value based on execution of TPR formula (double).
        '''

        tpr = ''
        tprVal = 0.0
        
        if height < 6.0:
            self._logChn.logMsg('Height passed to calcTPR below minimum of 6, cannot process (ht: %s)' % height)
            return (tpr, tprVal)
        
        # Perform TPR caclulation based on cover type.
        if speciesType == self.whiteSpruceFirType:
            tprVal = 1.3 + 10.398053 + 0.324415 * (height - 1.3) \
              + 0.005999608 * math.log(brHeightAge) * brHeightAge \
              - 0.838036 * math.pow(math.log(brHeightAge), 2) \
              + 27.487397 * (height - 1.3) / brHeightAge \
              + 1.191405 * math.log(height - 1.3)

            if tprVal <= 6.05:
                tpr = 'U'
            elif tprVal > 6.05 and tprVal <= 10.55:
                tpr = 'F'
            elif tprVal > 10.55 and tprVal <= 15.55:
                tpr = 'M'
            elif tprVal > 15.55:
                tpr = 'G'

        elif speciesType == self.pineType:
            tprVal = 1.3 + 10.940796 + 1.675298 * (height - 1.3) \
               - 0.932222 * math.pow(math.log(brHeightAge), 2) \
               + 0.005439671 * math.log(brHeightAge) * brHeightAge \
               + 8.228059 * (height - 1.3) / brHeightAge \
               - 0.256865 * (height - 1.3) * math.log(height - 1.3)

            if tprVal <= 7.05:
                tpr = 'U'
            elif tprVal > 7.05 and tprVal <= 12.05:
                tpr = 'F'
            elif tprVal > 12.05 and tprVal <= 16.05:
                tpr = 'M'
            elif tprVal > 16.05:
                tpr = 'G'
         
        elif speciesType == self.decidType:
            tprVal = 1.3 + 17.0100096 + 0.878406 * (height - 1.3) \
               + 1.836354 * math.log(brHeightAge) \
               - 1.401817 * math.pow(math.log(brHeightAge), 2) \
               + 0.43743 * math.log(height - 1.3) / brHeightAge

            if tprVal <= 10.05:
                tpr = 'U'
            elif tprVal > 10.05 and tprVal <= 14.05:
                tpr = 'F'
            elif tprVal > 14.05 and tprVal <= 18.05:
                tpr = 'M'
            elif tprVal > 18.05:
                tpr = 'G'

        elif speciesType == self.blackSpruceLarchType:
            tprVal = 1.3 + 4.903774 + 0.811817 * (height - 1.3) \
               - 0.363756 * math.pow(math.log(brHeightAge), 2) \
               + 24.030758 * (height - 1.3) / brHeightAge \
               - 0.102076 * (height - 1.3) * math.log(height - 1.3)

            if tprVal <= 6.05:
                tpr = 'U'
            elif tprVal > 6.05 and tprVal <= 7.05:
                tpr = 'F'
            elif tprVal > 7.05 and tprVal <= 10.05:
                tpr = 'M'
            elif tprVal > 10.05:
                tpr = 'G'

##        print '%s %s %s %s %s' % (speciesType, height, brHeightAge, tprVal, tpr)
            
        return (tpr, tprVal)
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def getAgeAndSpeciesType(self, sp, origin, curBaseYear):
        '''
        Returns the breat height age and species type so that a TPR
        calculation can be made.

        Parameters:
            sp - AVI species code.

            origin - origin of sp

            curBaseYear - normally the PHOTO_YR used when interpreting AVI.

        Returns:
            a tuple containing Breast Height Age and Species Type.
        '''
        

        # Make breast age origin adjustment based on type.
        # Also assign species to cover group to be used to
        # calculate the TPR.
        if sp.upper() in ('AW', 'BW', 'PB', 'A'):
            brHeightAge = curBaseYear - origin - self.AAdjustment
            speciesType = self.decidType
        elif sp.upper() in ('SW', 'SE', 'FD', 'FB', 'FA'):
            brHeightAge = curBaseYear - origin - self.SwAdjustment
            speciesType = self.whiteSpruceFirType
        elif sp.upper() in ('SB', 'LT', 'LA', 'LW'):
            brHeightAge = curBaseYear - origin - self.SbAdjustment
            speciesType = self.blackSpruceLarchType
        elif sp.upper() in ('P', 'PL', 'PJ', 'PA', 'PF'):
            brHeightAge = curBaseYear - origin - self.PAdjustment
            speciesType = self.pineType

        return (brHeightAge, speciesType)
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def getTPR_UsingPhotoYr(self, aviRec, aviLayer):
        '''
        Calculates a TPR value for either the overstorey or understorey record
        for AVI stand.
        
        Parameters
            aviRec - Record read for either overstorey or understorey

            aviLayer - Indicates which layer of AVI stand record is for.
                       Will be either OVER or UNDER
        Returns
            TPR if it could be calculated, an empty string '', or 'HT' if the
            value could not be calculated because height was below minimum
            requirements.
            
        '''

        tpr = ''

        # Make sure all required attributes for TPR calculation are present.
        sp = aviRec['sp1']
        if not sp:
            return ''
        
        height = aviRec['height']
        if height < 6:
            return ''
        
        origin = aviRec['origin']
        if not origin:
            return ''

        curBaseYear = aviRec['photo_yr']
        
        # The decade adjust determines how to calculate the origin
        # when determining age. If a generic decade is used then will
        # take mid-point of decade as origin year since the origin can
        # span 10 years. We assume it is a decade origin if the origin
        # ends in 0. If does not then assume it is an absolute origin
        # and do not adjust it.
        originStr = str(origin)
        if originStr.endswith('0'):
            origin += self.adjustDecadeYrs

        brHeightAge,speciesType = self.getAgeAndSpeciesType(sp, origin, curBaseYear)
                    
        # Make sure that breast height origin is usable.
        # If TPR cannot be calculated because layer is too young
        # then return flag to indicate this.
        if brHeightAge < 1:
            return ''

        tpr,tprVal = self.calcTPR(speciesType, height, brHeightAge)

        return tpr
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def accumulateStatistics(self, oldTpr, newTpr):
        '''
        Accumulate statistics to track changes to TPR from the old
        version to the calculated version.

        Parameters:
            oldTpr - original TPR

            newTpr - calculated TPR.
        '''

        # Accumulate statistics
        if oldTpr:
            if oldTpr != newTpr:
                if oldTpr in self.changeDict:
                    tprDict = self.changeDict[oldTpr]
                else:
                    tprDict = {}
                if newTpr in tprDict:
                    tprCnt = tprDict[newTpr]
                else:
                    tprCnt = 0
                tprCnt += 1
                tprDict[newTpr] = tprCnt
                self.changeDict[oldTpr] = tprDict
                
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def updateTPR_UsingPhotoYr(self, fcPath, readOnly=False):
        '''
        Update the TPR for both overstorey and understorey

        Parameters:
            fcPath = path to AVI feature class or table.

                
        '''
        
        # Ensure all required fields are present
        allFieldsFound = self.requiredFieldsPresent(fcPath)
        if not allFieldsFound:
            return False

        aviObj = AVI_Record(fcPath)
        
        # Make sure that the required fields are present.
        if readOnly:
            fieldList = aviObj.allFieldNames
            oidFieldName = ''
        else:
            fieldList = self.getRequiredFields(fcPath)
            oidFieldName = fieldList[-1]

        if readOnly:
            self._logChn.logMsg('\n***\n*** Testing AVI TPR values using PHOTO_YR...\n***')
        else:
            self._logChn.logMsg('\n***\n*** Updating AVI TPR values using PHOTO_YR...\n***')
        
        rsObj = srd_recordset.SRD_Recordset(fcPath)
        
        selCnt = int(arcpy.GetCount_management(fcPath).getOutput(0))
        arcpy.SetProgressor('step', 'Updating TPR...', 0, selCnt, 1)

        oTprIdx = fieldList.index('tpr')
        uTprIdx = fieldList.index('utpr')
        oDataIdx = fieldList.index('data')
        uDataIdx = fieldList.index('udata')
        oDataYrIdx = fieldList.index('data_yr')
        uDataYrIdx = fieldList.index('udata_yr')
        
        with arcpy.da.UpdateCursor(fcPath, fieldList) as cursor:
            for row in cursor:

                # Start with an empty AVI record then add values for required fields
                aviObj.fieldValues = AVI_RecordUtils.blankAVIRecord()
                
                reqFieldValues = rsObj.getRecord(row, fieldList, replaceNulls=True, stripBlanks=True)
                for fieldName in fieldList:
                    aviObj.fieldValues[fieldName] = reqFieldValues[fieldName]

                if readOnly:
                    oid = 0
                else:
                    # Need common field name for Object ID
                    aviObj.fieldValues['oid'] = reqFieldValues[oidFieldName]
                    oid = reqFieldValues[oidFieldName]

                # Get display information if performing tests
                if readOnly:
                    fieldList = aviObj.allFieldNames
                    fieldDisplayLenDict = AVI_RecordUtils.fieldDisplayLength()
                    blankAVIRecDict = AVI_RecordUtils.blankAVIRecord()
                    oFieldNames = aviObj.overstoryFieldNames[:]
                    uFieldNames = aviObj.understoryFieldNames[:]
                    oAviRec = aviObj.buildFormattedRecString(oFieldNames, fieldDisplayLenDict, blankAVIRecDict)
                    uAviRec = aviObj.buildFormattedRecString(uFieldNames, fieldDisplayLenDict, blankAVIRecDict)

                photoYr = aviObj.fieldValues['photo_yr']

                # Make sure that any photo year is valid.
                # Cannot be greater than the current year.
                # If the PHOT_YR is invlaid then the TPR will
                # be cleared to indicate it could not be calculated.
                curDate = datetime.date.today()
                curYear = curDate.year
                if photoYr:
                    if photoYr < 1975 or photoYr > curYear:
                        self._logChn.logMsg('\n***\n*** 11.1.11 Invalid PHOTO_YR: %s, must be between 1980 and current year (OID: %s)\n***' % (photoYr, oid))
                        photoYr = 0
                else:
                    photoYr = 0

                # Calculate for overstorey
                
                aviRec = aviObj.returnLayerRecord('OVER')
                oHt = aviRec['height']
                aviRec['photo_yr'] = photoYr
                oSp1 = aviRec['sp1']
                oNFL = aviRec['nfl']
                oDataCode = aviRec['data']
                oTPR_original = aviRec['tpr']
                oTPR = oTPR_original
                oAnthVeg = aviRec['anth_veg']
                oNatNon = aviRec['nat_non']
                oDataYr = aviRec['data_yr']
                
                oStruct = aviRec['struc']

                if photoYr:
                    if oHt > 5:                    
                        oTPR = self.getTPR_UsingPhotoYr(aviRec, 'OVER')
                        if VERBOSE:
                            self._logChn.logMsg('Overstorey TPR Old: %s New: %s (OID: %s)' % (oTPR_original, oTPR, oid))
                        if oTPR:
                            if oDataCode == 'I':
                                oDataCode = ''
                                oDataYr = 0
                        else:
                            oTPR = oTPR_original
                            if VERBOSE:
                                self._logChn.logMsg('Overstorey TPR Old: %s New: %s (OID: %s)' % (oTPR_original, oTPR, oid))
                            ##oDataCode = 'I'
                else:
                    oTPR = oTPR_original
                    ##oTPR = ''
                    
                # Calculate for understorey
                aviRec = aviObj.returnLayerRecord('UNDER')
                uHt = aviRec['height']
                aviRec['photo_yr'] = photoYr
                uSp1 = aviRec['sp1']
                uNFL = aviRec['nfl']
                uDataCode = aviRec['data']
                uTPR_original = aviRec['tpr']
                uTPR = uTPR_original
                uAnthVeg = aviRec['anth_veg']
                uNatNon = aviRec['nat_non']
                uDataYr = aviRec['data_yr']

                if photoYr:
                    if uHt > 5:
                        uTPR = self.getTPR_UsingPhotoYr(aviRec, 'UNDER')
                        if VERBOSE:
                            self._logChn.logMsg('Understorey TPR Old: %s New: %s (OID: %s)' % (uTPR_original, uTPR, oid))                        
                        if uTPR:
                            if uDataCode == 'I':
                                uDataCode = ''
                                uDataYr = 0
                        else:
                            uTPR = uTPR_original
                            if VERBOSE:
                                self._logChn.logMsg('Understorey TPR Old: %s New: %s (OID: %s)' % (uTPR_original, uTPR, oid))                        
                            ##uDataCode = 'I'
                else:
                    uTPR = uTPR_original
                    ##uTPR = ''

                # Enforce AVI specifications where we can.

                if oStruct == 'M':                        
                    # Where the overstorey and understorey species are the same
                    # then the TPR is taken from the overstorey.
                    if oTPR and oSp1:
                        if oSp1 == uSp1:
                            uTPR = oTPR
                        
                    # When there is a forested over non-forest vegetated then
                    # the TPR will be taken from the overstorey.                
                    if uNFL and oTPR and oSp1:
                        uTPR = oTPR
                    if uAnthVeg == 'CPR':
                        if oTPR and oSp1:
                            uTPR = oTPR
                    if uNatNon == 'NMB':
                        if oTPR and oSp1:
                            uTPR = oTPR
                        
                    if oNFL and uTPR and uSp1:
                        oTPR = uTPR
                    if oAnthVeg == 'CPR':
                        if uTPR and uSp1:
                            oTPR = uTPR
                    if oNatNon == 'NMB':
                        if uTPR and uSp1:
                            oTPR = uTPR
                    
                row[oTprIdx] = oTPR
                ##row[oDataIdx] = oDataCode
                ##row[oDataYrIdx] = oDataYr
                row[uTprIdx] = uTPR
                ##row[uDataIdx] = uDataCode
                ##row[uDataYrIdx] = uDataYr

                self.accumulateStatistics(oTPR_original, oTPR)
                self.accumulateStatistics(uTPR_original, uTPR)
                
                if readOnly:

                    aviObj.fieldValues = rsObj.getRecord(row, fieldList, replaceNulls=True, stripBlanks=True)
                    oNewAviRec = aviObj.buildFormattedRecString(oFieldNames, fieldDisplayLenDict, blankAVIRecDict)
                    uNewAviRec = aviObj.buildFormattedRecString(uFieldNames, fieldDisplayLenDict, blankAVIRecDict)
                    
                    self._logChn.logWarning('--------------------------------------------------\n')
                    self._logChn.logWarning(oAviRec)
                    self._logChn.logWarning(uAviRec)
                    self._logChn.logWarning('\n***\n***    Overstorey TPR Old: %s New: %s\n***' % (oTPR_original, oTPR))
                    self._logChn.logWarning('***    Overstorey Data Code %s\n***' % oDataCode)
                    self._logChn.logWarning('***\n***    Understorey TPR Old: %s New: %s\n***' % (uTPR_original, uTPR))
                    self._logChn.logWarning('***    Understorey Data Code %s\n***\n' % uDataCode)
                    self._logChn.logWarning(oNewAviRec)
                    self._logChn.logWarning(uNewAviRec)
                    self._logChn.logWarning('\n--------------------------------------------------')
                else:
                    cursor.updateRow(row)
                        
                arcpy.SetProgressorPosition()
                
        arcpy.ResetProgressor()

        self._logChn.logMsg('\n***\n*** TPR update complete\n***')

        # Print the change statistics.
        if self.changeDict:
            self._logChn.logMsg('\n***\n*** TPR Change Statistics\n***\n')
            tprList = self.changeDict.keys()
            tprList.sort()
            for tpr in tprList:
                self._logChn.logMsg('Original TPR: %s' % tpr)
                tprDict = self.changeDict[tpr]
                tprList2 = tprDict.keys()
                tprList2.sort()
                for tpr2 in tprList2:
                    self._logChn.logMsg('   To %s: %s' % (tpr2, tprDict[tpr2]))

    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def validateTPR_Record(self, aviRecObj):
        '''
        Validates the TPR for an AVI record looking at both the overstorey
        and understorey.

        Parameters:
            aviRecord - AVI record to be evaluated.

        Returns:
            List of errors if any were encountered otherwise an empty list.
        '''

        aviRecord = aviRecObj.fieldValues

        errList = []

        height = aviRecord['height']
        sp1 = aviRecord['sp1']
        tpr = aviRecord['tpr']
        data = aviRecord['data']

        struc = aviRecord['struc']
        
        uheight = aviRecord['uheight']
        usp1 = aviRecord['usp1']
        utpr = aviRecord['utpr']
        udata = aviRecord['udata']

        # PHOTO_YR is required to perform TPR calculation.
        if 'photo_yr' in aviRecord:
            photoYr = aviRecord['photo_yr']
            if not photoYr:
                errList.append('11.1.1 Field PHOTO_YR not populated, required to calculate TPR')
                return errList
        else:
            errList.append('11.1.1 Field PHOTO_YR missing, required to calculate TPR')
            return errList
        
        # Perform TPR checks for overstorey.
        errList += self.validateLayerTPR(aviRecObj, 'OVER')

        if struc == 'M':

            # How the understorey TPR is evaluated depends on the
            # stand sturcture. If it is M then the understorey is
            # only calculated if the overstorey/understorey SP1 are
            # not the same.
            if sp1 and sp1 != usp1:
                errList += self.validateLayerTPR(aviRecObj, 'UNDER')
                
            # Perform relational checks if each layer has a TPR.
            if tpr and utpr:
                # Forested over forested stand
                if sp1 and usp1:
                    # Make sure the understorey TPR is valid before comparing to overstorey.
                    if not utpr in ['G','M','F','U','']:
                        msg = 'UNDER: Invalid TPR: %s, must be one of G, M, F, or U' % utpr
                        errList.append(msg)
                    else:
                        if sp1 == usp1 and tpr != utpr:
                            msg = '11.1.2 Overstorey and understorey have different TPR values, should be same when overstorey species 1 is same as understorey species 1. (O sp1: %s tpr: %s U sp1: %s tpr: %s)' % (sp1, tpr, usp1, utpr)
                            errList.append(msg)

                else:
                    # Non-forested over non-forested stand.
                    if not sp1 and not usp1:
                        if utpr and tpr != utpr:
                            msg = '11.1.3 Non-forest Overstorey/Non-forest Understorey: TPR values do not match. (O tpr: %s U tpr: %s)' % (tpr, utpr)
                            errList.append(msg)
                    else:
                        # Forest/non-forest layer.
                        if sp1:
                            if utpr and tpr != utpr:
                                msg = '11.1.4 Understorey TPR value should match forested overstorey TPR. (O tpr: %s U tpr: %s)' % (tpr, utpr)
                                errList.append(msg)
                            
                        elif usp1:
                            if tpr and tpr != utpr:
                                msg = '11.1.5 Overstorey TPR value should match forested understorey TPR. (O tpr: %s U tpr: %s)' % (tpr, utpr)
                                errList.append(msg)
        else:
            errList += self.validateLayerTPR(aviRecObj, 'UNDER')
            
        return errList
        
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def validateLayerTPR(self, aviRecObj, aviLayer):
        '''
        Performs the TPR validation on the Overstorey or Understorey layer

        Parameters:
            aviRecord - AVI attributes

            aviLayer - the layer being validated, wither OVER or UNDER
        '''

        # NOTE
        # Because of changes to way TPR is generated we are no longer concerned with
        # having an I in the DATA field to flag Interpreted TPRs. It will be considered
        # implied since we will be calculating all stands over 5 meters.
        aviRecord = aviRecObj.fieldValues
        
        if aviLayer == 'OVER':
            sp = aviRecord['sp1']
            height = aviRecord['height']
            tpr = aviRecord['tpr']
            data = aviRecord['data']
            nfl = aviRecord['nfl']
            anth_veg = aviRecord['anth_veg']
            nat_non = aviRecord['nat_non']
        else:
            sp = aviRecord['usp1']
            height = aviRecord['uheight']
            tpr = aviRecord['utpr']
            data = aviRecord['udata']
            nfl = aviRecord['unfl']
            anth_veg = aviRecord['uanth_veg']
            nat_non = aviRecord['unat_non']

        struc = aviRecord['struc']
        
        photoYr = aviRecord['photo_yr']
        
        errList = []

        # Remove any possible imbedded spaces.
        tpr = tpr.strip()

        # If TPR is not valid then just return with error message.
        if not tpr in ['G','M','F','U','']:
            msg = '%s: 2.10 Invalid TPR: %s, must be one of G, M, F, or U' % (aviLayer, tpr)
            errList.append(msg)
            return errList
        
        # Stands greater than 5 meters must have a calculated TPR if possible.
        if height > 5:

            # Do not worry about having an I for interpreted TPR since will be cleared
            # when the actual calculation is performed.
            layerRec = aviRecObj.returnLayerRecord(aviLayer)
            layerRec['photo_yr'] = photoYr

            calcTpr = self.getTPR_UsingPhotoYr(layerRec, aviLayer)

            # If a TPR was submitted then compare to calculated TPR.
            if tpr:
                if calcTpr:
                    # Compare TPR submitted to that calculated.
                    if calcTpr != tpr:
                        errList.append('%s: 11.1.12 Calculated TPR: %s does not match submitted TPR: %s' % (aviLayer, calcTpr, tpr))

        if sp and not tpr:
            msg = '%s: 11.1.6 TPR missing from forested layer' % aviLayer
            errList.append(msg)
            
        if nfl and not tpr:
            msg = '%s: 11.1.7 TPR missing: when NFL populated: (%s) then TPR is required' % (aviLayer, nfl)
            errList.append(msg)

        if anth_veg == 'CPR' and not tpr:
            msg = '%s: 11.1.8 TPR missing: when Anthropogenic Vegetated is CPR then TPR is required' % aviLayer
            errList.append(msg)

        if nat_non:
            if nat_non == 'NMB':
                if not tpr:
                    msg = '%s: 11.1.9 TPR missing: when Naturally Non-vegetated is NMB then TPR is required' % aviLayer
                    errList.append(msg)
            else:
                if tpr:
                    errList.append('%s 11.1.10 TPR cannot be present in Naturally Non-vegetated layer unless NMB (NatNon: %s TPR: %s)' % (aviLayer, nat_non, tpr))

        return errList
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def validateTPR(self, fcPath):
        '''
        Performs validation of TPR values of AVI feature class.
        And error and summary table will be generated containing
        the results of the validation process.

        Parameters:
            fcPath - path to AVI feature class to validate.
        '''

        # Ensure all required fields are present
        allFieldsFound = self.requiredFieldsPresent(fcPath)
        if not allFieldsFound:
            return False
        
        # Make sure that the required fields are present.
        fieldList = self.getRequiredFields(fcPath)
        oidFieldName = fieldList[-1]

        # Count maintained for errors and warnings,.
        self.totalErrorCount = 0
        self.totalWarningCount = 0
        
        self.aviFCPath = fcPath

        utilFCObj = srd_featureclass_info.SRD_FeatureClassInfo(fcPath)

        self.aviFCName = os.path.basename(fcPath)
        self.fgdbPath = utilFCObj.getFGDBPath()
        if not self.fgdbPath:
            raise SRD_Exception('Cannot obtain path to FGDB containing feature class')

        self.fgdbDir = utilFCObj.getParentDir()
        self.fgdbName = os.path.basename(self.fgdbPath)

        name,ext = os.path.splitext(self.fgdbName)
        logFileName = '%s_tpr.log' % name
        self.logFilePath = os.path.join(self.fgdbDir, logFileName)
        self._logChn = srd_logging.SRD_Log(self.logFilePath)

        self.auditTableName = '%s_tpr_log' % self.aviFCName
        self.auditTablePath = os.path.join(self.fgdbPath, self.auditTableName)

        self.infoTableName = '%s_tpr_info' % self.aviFCName
        self.infoTablePath = os.path.join(self.fgdbPath, self.infoTableName)

        aviRecObj = AVI_Record(self.aviFCPath)

        auditLogObj = srd_audit_log.SRD_AuditLog(self.auditTablePath , self.infoTablePath)
        auditLogObj.deleteAllTables()

        rsObj = srd_recordset.SRD_Recordset(self.aviFCPath)
        
        selCnt = int(arcpy.GetCount_management(self.aviFCPath).getOutput(0))
        arcpy.SetProgressor('step', 'Validating TPR...', 0, selCnt, 1)

        tprObj = AVI_TPR()

        with arcpy.da.SearchCursor(self.aviFCPath, fieldList) as cursor:
            for row in cursor:

                # Start with an empty AVI record then add values for required fields
                aviRecObj.fieldValues = AVI_RecordUtils.blankAVIRecord()
                
                reqFieldValues = rsObj.getRecord(row, fieldList, replaceNulls=True, stripBlanks=True)
                for fieldName in fieldList:
                    aviRecObj.fieldValues[fieldName] = reqFieldValues[fieldName]
                # Need common field name for Object ID
                aviRecObj.fieldValues['oid'] = reqFieldValues[oidFieldName]
                
                errList = tprObj.validateTPR_Record(aviRecObj)
                if errList:
                    self.totalErrorCount += len(errList)
                    auditLogObj.addAuditRecords(aviRecObj.fieldValues['oid'], errList, 'ERROR')        
                
                arcpy.SetProgressorPosition()
                
        arcpy.ResetProgressor()

        auditLogObj.writeAuditRecords()
        
        # Add basic audit information for report purposes.
        infoList = []
        infoList.append('AVI TPR Audit Report')
        infoList.append('QC Software Version: %s' % self.version())
        infoList.append('AVI Feature Class: %s' % self.aviFCPath)
        infoList.append('Date: %s' % datetime.date.today())
        infoList.append('Features Processed: %s' % selCnt)
        infoList.append('Total Errors: %s' % self.totalErrorCount)
        infoList.append('Total Warnings: %s' % self.totalWarningCount)
        infoList.append('Audit Run By: %s' % os.environ.get('USERNAME'))
        auditLogObj.addInformationMessages(infoList)
        auditLogObj.writeInformationMessages()

        # Delete and pre-existing summaries.
        errorSummaryPath = os.path.join(self.fgdbPath, 'tpr_error_summary')
        if arcpy.Exists(errorSummaryPath):
            arcpy.Delete_management(errorSummaryPath)
        warningSummaryPath = os.path.join(self.fgdbPath, 'tpr_warning_summary')
        if arcpy.Exists(warningSummaryPath):
            arcpy.Delete_management(warningSummaryPath)

        self._logChn.logMsg('\n *** Errors: %s Warnings: %s' % (self.totalErrorCount, self.totalWarningCount))
        self.buildSummaryTables(self.auditTablePath, errorSummaryPath, warningSummaryPath)
                
        self._logChn.logMsg('\n *** Any Errors or Warnings have been logged to: %s\n***' % self.auditTablePath)
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    def buildSummaryTables(self, srcTable, errorSummaryPath, warningSummaryPath):
        '''
        Build the Error and Warning summary tables. This is used to
        replace the frequency command so that we can eliminate the need
        for an Advanced License to run this tool.

        Parameters:
            srcTable - path to table containing the error and warnings.

            errorSummaryPath - path to the error summary table to create.

            warningSummaryPath - path to the warning summary table to create.
        '''

        if not arcpy.Exists(srcTable):
            self._logChn.logMsg('\n***\n*** No audit table to process\n***')
            return False
        
        # Build dictionaries based on errors and warnings to accumulate counts of each message type.
        warnDict = {}
        errorDict = {}
        with arcpy.da.SearchCursor(srcTable, ['TYPE', 'MESSAGE']) as cursor:
            for row in cursor:
                msgType = row[0]
                msg = row[1]
                if msgType.upper() == 'ERROR':
                    if msg in errorDict:
                        cnt = errorDict[msg]
                    else:
                        cnt = 0
                    cnt += 1
                    errorDict[msg] = cnt
                elif msgType.upper() == 'WARNING':
                    if msg in warnDict:
                        cnt = warnDict[msg]
                    else:
                        cnt = 0
                    cnt += 1
                    warnDict[msg] = cnt

        
        # Populate the error summary table
        fgdbPath = os.path.dirname(errorSummaryPath)
        errTblName = os.path.basename(errorSummaryPath)
        if arcpy.Exists(errorSummaryPath):
            arcpy.Delete_management(errorSummaryPath)
        arcpy.CreateTable_management(fgdbPath, errTblName)
        arcpy.AddField_management(errorSummaryPath, 'COUNT', 'LONG')
        arcpy.AddField_management(errorSummaryPath, 'MESSAGE', 'TEXT', '', '', 250)
        outChn = arcpy.da.InsertCursor(errorSummaryPath, ('COUNT', 'MESSAGE'))
        msgList = errorDict.keys()
        msgList.sort()
        for msg in msgList:
            cnt = errorDict[msg]
            outChn.insertRow([cnt, msg])

        # Populate the error summary table
        fgdbPath = os.path.dirname(warningSummaryPath)
        warnTblName = os.path.basename(warningSummaryPath)
        if arcpy.Exists(warningSummaryPath):
            arcpy.Delete_management(warningSummaryPath)
        arcpy.CreateTable_management(fgdbPath, warnTblName)
        arcpy.AddField_management(warningSummaryPath, 'COUNT', 'LONG')
        arcpy.AddField_management(warningSummaryPath, 'MESSAGE', 'TEXT', '', '', 250)
        outChn = arcpy.da.InsertCursor(warningSummaryPath, ('COUNT', 'MESSAGE'))
        msgList = warnDict.keys()
        msgList.sort()
        for msg in msgList:
            cnt = warnDict[msg]
            outChn.insertRow([cnt, msg])
            


# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
def testTPR2():
    ''' 
    Used to test individual TPR calculations to confirm results.
    '''

##    speciesType = tprObj.whiteSpruceFirType
    speciesType = tprObj.decidType
##    speciesType = tprObj.pineType
##    speciesType = tprObj.blackSpruceLarchType
    
    ht = 12
    brAge = 2005 - 1950
    tpr,tprVal = tprObj.calcTPR(speciesType, ht, brAge)
    print '%s %s %s %s' % (ht, brAge, tpr, tprVal)
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
def testTPRAudit():

    fcPathArg = r'D:\test Data\TPR\aviTest.gdb\avi1'
    

    myObj = AVI_TPR()

    aviRec = AVI_Record(fcPathArg)

    fieldList = aviRec.allFieldNames[:]

    oidFieldName = arcpy.Describe(fcPathArg).OIDFieldName
    fieldList.append(oidFieldName)
    
    rsObj = srd_recordset.SRD_Recordset(fcPathArg)
    
    with arcpy.da.SearchCursor(fcPathArg, fieldList) as cursor:
        for row in cursor:

            # Get a dictionary or values to enable easy access.
            aviRec.fieldValues = rsObj.getRecord(row, fieldList, replaceNulls=True, stripBlanks=True)
            aviRec.fieldValues['oid'] = aviRec.fieldValues[oidFieldName]
            
            errList = myObj.validateTPR(aviRec)
            for err in errList:
                print err
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
def testTPRUpdate():

    fcPathArg = r'D:\test Data\TPR\aviTest.gdb\avi1'
    fcPathArg = r'D:\test Data\TPR\aviTest.gdb\tst1'
    fcPathArg = r'D:\test Data\TPR\aviTest.gdb\val_tst1'
    ##fcPathArg = r'D:\test Data\TPR\aviTest.gdb\tst2'
    ##fcPathArg = r'D:\test Data\TPR\tpr_tests.gdb\tst_template'
    ##fcPathArg = r'D:\test Data\TPR\AVI_TPR_TEST.gdb\tst1'
    
    outFcPathArg = '%s_update' % fcPathArg
    arcpy.Copy_management(fcPathArg, outFcPathArg)

    myObj = AVI_TPR()

    myObj.updateTPR_UsingPhotoYr(outFcPathArg)

# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
def tst():

    tprObj = AVI_TPR()
    fcPathArg = r'D:\test Data\TPR\aviTest.gdb\val_tst1_update'
    tprObj.validateTPR(fcPathArg)
    
# ======================================================================
if __name__ == '__main__':
    
    arcpy.env.overwriteOutput = True
    # If using geographic features then may be best to modify these.
    arcpy.env.XYTolerance = .001
    arcpy.env.XYResolution = .0001
    arcpy.gp.logHistory = False
    
    tprObj = AVI_TPR()

    fcPathArg = r'D:\testData\TPR\aviTest.gdb\avi1'

    # Forested over NFL test
    fcPathArg = r'D:\testData\TPR\aviTest.gdb\sp_nfl'

    

    # Complex stand test
##    fcPathArg = r'D:\testData\TPR\aviTest.gdb\complex'

    # I data in overstorey test
##    fcPathArg = r'D:\testData\TPR\aviTest.gdb\o_data'

    # I data in understorey test
##    fcPathArg = r'D:\testData\TPR\aviTest.gdb\u_data'
    

    testTPRUpdate()
    
    outFcPathArg = r'\\goa\shared\AF\FERD\FMB\Spatial_Proj\s10\data\avi_compare\s10_avie_inventories.gdb\AVIE_2018'
    ##tprObj.updateTPR_UsingPhotoYr(outFcPathArg)
    

    fcPathArg = r'D:\test Data\TPR\aviTest.gdb\avi1'
    fcPathArg = r'D:\test Data\TPR\aviTest.gdb\avi1_update'
##    fcPathArg = r'D:\test Data\TPR\aviTest.gdb\tst1_update'
##    fcPathArg = r'D:\test Data\TPR\aviTest.gdb\complex'
    fcPathArg = r'D:\test Data\TPR\aviTest.gdb\val_tst1'
##    fcPathArg = r'D:\test Data\TPR\aviTest.gdb\val_tst1_update'

##    tprObj.validateTPR(fcPathArg)

    ##import cProfile
    ##cProfile.run('tst()')
