OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

sarif message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: RE: [sarif] #401: Improved design of address object design


I appreciate you coming up with a nice idea that avoids Nullables, at the expense of one more property.

 

When Michael and I talked about it just now, we finally conceded that we have been tying ourselves in knots trying to avoid Nullables where they would naturally occur, and the best thing to do was just let them into the SDK.

 

Would you agree that in the presence of Nullables, they are preferable to adding the new property?

 

Larry

 

From: James Kupsch <kupsch@cs.wisc.edu>
Sent: Monday, April 29, 2019 2:19 PM
To: Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>; Michael Fanning <Michael.Fanning@microsoft.com>; OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org>
Subject: Re: [sarif] #401: Improved design of address object design

 

If you add a relativeToAddressIndex, then instead of making it relative to the top-most parent, you can make it relative to this object, and -1 works here as it can indicate unknown.  This also allows two addresses (only if the relativeToAddressIndex are identical and not -1) to be compared, and allows relativeness to other containers than the top most which might be more relevant. Below is an example. Empty values are unknown.

name: STACK1
kind: stack
length: -300000
offsetFromParent:
parentIndex:
relativeAddress:
relativeToAddressIndex:
absoluteAddress: 1250000

name: STACK1.frame4
kind: stackFrame
length: -2000
offsetFromParent: -1000
parentIndex: 0
relativeAddress: -1000
relativeToAddressIndex: 0
absoluteAddress: 1249000

name: variable444
kind: intData
length: 8
offsetFromParent: 32
parentIndex: 1
relativeAddress: -968
relativeToAddressIndex: 0
absoluteAddress: 1249030

On 4/29/19 3:20 PM, Larry Golding (Myriad Consulting Inc) wrote:

Thereâs a problem with relativeAddress, though. Hereâs what I currently have:

3.32.4 Relative address calculation

Each address object has an associated value called its ârelative addressâ which is the offset of the address from the address of the top-most object in its parent chainâ

3.32.7 relativeAddress property

If parentIndex (Â3.32.13) is present, an address object MAY contain a property named relativeAddress whose value is an integer containing the relative address (see Â3.32.4) of thisObject.

If parentIndex is absent, then relativeAddressSHALL be absent.

If relativeAddress is absent, it SHALL default to -1, which is otherwise not a valid value for this property.

 

Thatâs wrong. If your top-most address is a downward-growing stack, relativeAddress can be negative. And unlike offsetFromParent, we canât say (I mean, we can, but donât think we should) that relativeAddress SHALL be present if parentIndex is present, so we canât use 0 as the default.

 

Ideas?

 

Iâm going to have lunch now. Back in a while.

 

Larry

 

From: sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> On Behalf Of Larry Golding (Myriad Consulting Inc)
Sent: Monday, April 29, 2019 12:59 PM
To: Michael Fanning <Michael.Fanning@microsoft.com>; James Kupsch <kupsch@cs.wisc.edu>; OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org>
Subject: RE: [sarif] #401: Improved design of address object design

 

I approve this message! Much better than my solution.

 

From: Michael Fanning <Michael.Fanning@microsoft.com>
Sent: Monday, April 29, 2019 12:56 PM
To: James Kupsch <kupsch@cs.wisc.edu>; Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>; OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org>
Subject: RE: [sarif] #401: Improved design of address object design

 

Hereâs my suggestion:

 

For address.length, use a sentinel value of â0â meaning ânot providedâ. This makes sense, as Jim noted, a 0 length memory address isnât sensible. You donât posit describing an insertion point in memory that shoves everything to the right. 😊

 

For address.offsetFromParent, this property should only be consulted in cases where address.parentIndex >= 0. If address.parentIndex is -1, you ignore the offsetFromParent value (in all cases). If address.parentIndex >= 0, the address.offsetFromParent must be populated (and it is fine if its value is 0).

 

This requires producers to always provided offsetFromParent when using the address parenting chain. IOW, you canât describe your parenting relationship and depend strictly on relativeAddress and/or absoluteAddress to render those final computed values. I think this trade-off is ok, particularly as compared to consulting a separate Boolean in order to know whether youâre going backward in memory.

 

Michael

From: James Kupsch <kupsch@cs.wisc.edu>
Sent: Monday, April 29, 2019 12:46 PM
To: Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>; Michael Fanning <Michael.Fanning@microsoft.com>; OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org>
Subject: Re: [sarif] #401: Improved design of address object design

 

The unknown is only for this address to its parents.  The same problem exists whether you use -1 or unknownOffset/Length. It is useful to know offsets up to the point of where it is unknown as you at least know it is relative to this address (grand..)parent object.  The cleaner approach to working around the SDK not supporting nullable is to use the extra boolean.

On 4/29/19 2:18 PM, Larry Golding (Myriad Consulting Inc) wrote:

I really think my previous suggestion of growthDirection is the least bad alternative if we decide NOT to use Nullable<int>. Iâd modify my proposal to a Boolean growsTowardLowMemory, default false.

 

From: sarif@lists.oasis-open.org <sarif@lists.oasis-open.org> On Behalf Of Larry Golding (Myriad Consulting Inc)
Sent: Monday, April 29, 2019 11:46 AM
To: James Kupsch <kupsch@cs.wisc.edu>; Michael Fanning <Michael.Fanning@microsoft.com>; OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org>
Subject: RE: [sarif] #401: Improved design of address object design
Importance: High

 

Iâm not as happy as I was a few moments ago. If the âunknownâ properties default to false, then if you donât populate offsetFromParent (which happens at the top of every parent chain) or length (which can happen anywhere along the parent chain), then you have to remember to set the corresponding âunknownâ property. You wonât. Of if the âunknownâ properties default to true, you have to set them to false whenever you do populate offsetFromParent or length. You wonât.

 

Michael (addressing this question specifically to him because of the SDK impact), I donât see a good alternative to Nullable<int>. Please let us know your opinion. Raising to red-bang.

 

Larry

 

From: Larry Golding (Myriad Consulting Inc)
Sent: Monday, April 29, 2019 11:37 AM
To: James Kupsch <kupsch@cs.wisc.edu>; Michael Fanning <Michael.Fanning@microsoft.com>; OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org>
Subject: RE: [sarif] #401: Improved design of address object design

 

I could accept Boolean unknownLength and unknownOffsetFromParent, default false for both.

 

I will write it this way and send a draft. I can always change this aspect of the design if the TC objects.

 

Larry

 

-----Original Message-----
From: James Kupsch <kupsch@cs.wisc.edu>
Sent: Monday, April 29, 2019 11:30 AM
To: Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>; Michael Fanning <Michael.Fanning@microsoft.com>; OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org>
Subject: Re: [sarif] #401: Improved design of address object design

 

I think this alternative complicates the math for consumers, a boolean name unknownLength and unknownOffset would be better choices if there can't be a null value.  The default value could be the length not equal to the default length or offset (-1 would be good default).  Then length and offset can take any value, negative or positive.

 

Jim

 

 

 

On 4/29/19 12:45 PM, Larry Golding (Myriad Consulting Inc) wrote:

> As an alternative to allowing lengthand offsetFromParentto be negative,

> we could define a property growthDirection(or even just direction)

> values "positive"and "negative", default value "positive". If itâs

> "negative", then both offsetFromParentand lengthare interpreted as

> negative numbers.

>

> Iâm not 100% happy with that either â it would be too easy for an SDK

> consumer to add offsetFromParentto its parent address without noticing

> it should have reversed the sign.

>

> *From:*Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>

> *Sent:* Monday, April 29, 2019 10:25 AM

> *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>;

> James Kupsch <kupsch@cs.wisc.edu>; Michael Fanning

> <Michael.Fanning@microsoft.com>; OASIS SARIF TC Discussion List

> <sarif@lists.oasis-open.org>

> *Subject:* RE: [sarif] #401: Improved design of address object design

>

> Another question: What does a lengthof zero mean? Is it an âinsertion

> pointâ, a zero-length range preceding the location specified by

> offsetFromParent/absoluteAddress/relativeAddress?

>

> *From:*sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>

> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>> *On

> Behalf Of *Larry Golding (Myriad Consulting Inc)

> *Sent:* Monday, April 29, 2019 10:18 AM

> *To:* James Kupsch <kupsch@cs.wisc.edu <mailto:kupsch@cs.wisc.edu>>;

> Michael Fanning <Michael.Fanning@microsoft.com

> <mailto:Michael.Fanning@microsoft.com>>; OASIS SARIF TC Discussion List

> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>>

> *Subject:* RE: [sarif] #401: Improved design of address object design

>

> Sorry, I shouldnât have directed those questions only to Michael. It was

> just because he proposed the original redesign in #403. Of course I want

> to hear feedback from the entire TC.

>

> *From:*Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com

> <mailto:v-lgold@microsoft.com>>

> *Sent:* Monday, April 29, 2019 10:16 AM

> *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com

> <mailto:v-lgold@microsoft.com>>; James Kupsch <kupsch@cs.wisc.edu

> <mailto:kupsch@cs.wisc.edu>>; Michael Fanning

> <Michael.Fanning@microsoft.com <mailto:Michael.Fanning@microsoft.com>>;

> OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org

> <mailto:sarif@lists.oasis-open.org>>

> *Subject:* RE: [sarif] #401: Improved design of address object design

>

> Another concern I have is around length. I want to be able to write that

> if an addressobject occurs within a physicalLocationobject thisObject,

> and if thisObject.lengthand thisObject.address.lengthare both present,

> then then *SHALL* be equal.

>

> As the spec stands, I canât write this because physicalLocation.lengthis

> non-negative. I can add words like this:

>

> If an addressobject occurs within a physicalLocationobject thisObject,

> and if thisObject.lengthand thisObject.address.lengthare both present,

> then then *SHALL* be equal. In that case, if thisObject.address.lengthis

> negative, then thisObject.lengthwill also be negative. In all other

> circumstances, lengthSHALL be non-negative.

>

> *Michael*, are you ok with that complication?

>

> Larry

>

> *From:*sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>

> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>> *On

> Behalf Of *Larry Golding (Myriad Consulting Inc)

> *Sent:* Monday, April 29, 2019 10:08 AM

> *To:* James Kupsch <kupsch@cs.wisc.edu <mailto:kupsch@cs.wisc.edu>>;

> Michael Fanning <Michael.Fanning@microsoft.com

> <mailto:Michael.Fanning@microsoft.com>>; OASIS SARIF TC Discussion List

> <sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>>

> *Subject:* RE: [sarif] #401: Improved design of address object design

>

> Jim,

>

>  From an SDK standpoint we have a problem with negative values because

> without -1 as a sentinel value, we need to make the .NET type

> Nullable<int>instead of int. So far we have avoided nullables in the

> API. Our code generation doesnât currently support them, but it wouldnât

> be hard to add.

>

> *Michael*, could you weigh in on this? The .NET Framework Design

> Guidelines (Â8.8) say:

>

> *CONSIDER*using Nullable<T>to represent values that might not be present

> (/i.e./, optional values).

>

> **

>

> â and a quick web search doesnât turn up recommendations to avoid them

> in this scenario.

>

> Iâd just be concerned that this is the only place we have nullables. Are

> there alternatives?

>

> In the meantime Iâll write the words for the rest of Jimâs feedback.

>

> Larry

>

> *From:*James Kupsch <kupsch@cs.wisc.edu <mailto:kupsch@cs.wisc.edu>>

> *Sent:* Monday, April 29, 2019 8:40 AM

> *To:* Michael Fanning <Michael.Fanning@microsoft.com

> <mailto:Michael.Fanning@microsoft.com>>; Larry Golding (Myriad

> Consulting Inc) <v-lgold@microsoft.com <mailto:v-lgold@microsoft.com>>;

> OASIS SARIF TC Discussion List <sarif@lists.oasis-open.org

> <mailto:sarif@lists.oasis-open.org>>

> *Subject:* Re: [sarif] #401: Improved design of address object design

>

> For #2,I  was thinking the "stack" kind would be the whole stack.  Also

> having a "stackFrame" kind would also be useful as it also a defined

> memory region contained within the "stack" kind.

>

> JIm

>

> On 4/29/19 10:20 AM, Michael Fanning wrote:

>

> Right now, the effective address is absolute if unparented and relative

> to some other thing, if parented.

>

> If you chase to the root object, letâs say itâs a module, its effective

> address provides the ability to distinguish a childâs relative address

> vs. physical memory location.

>

> IOW, the childâs relative address is its offset + all offsets up to (but

> not including) the root object. If you add the effective address of the

> root object, you get the absolute memory address (if available).

>

> Make sense? What do you think?

>

> For #2 stack, is the kind a stack frame?

>

> For #3, I see the problem. Every address in a PLC has an associated

> region but the run.addresses cache does not provide this association.

> Let me have a word with Larry.

>

> *From:*sarif@lists.oasis-open.org <mailto:sarif@lists.oasis-open.org>

> <sarif@lists.oasis-open.org> <mailto:sarif@lists.oasis-open.org> *On

> Behalf Of *James Kupsch

> *Sent:* Monday, April 29, 2019 8:06 AM

> *To:* Larry Golding (Myriad Consulting Inc) <v-lgold@microsoft.com>

> <mailto:v-lgold@microsoft.com>; OASIS SARIF TC Discussion List

> <sarif@lists.oasis-open.org> <mailto:sarif@lists.oasis-open.org>

> *Subject:* Re: [sarif] #401: Improved design of address object design

>

> After reading the address section changes.  I have a couple of comments:

>

> 1) I would think that a common use of address would be something on the

> stack. To facilitate this, the offsetFromParent should be allowed to be

> negative as the "bottom" of stack is most logical place to be offset

> from, and since some stacks grow from higher to lower memory addresses,

> the offset is negative.  The use of -1 for unknown mechanism would have

> to be changed for this to work.

>

> 2) Add stack as a kind

>

> 3) There is a length in physical location, but not all addresses will

> appear in a physical location (probably just those without parentIndex)

> a length field would be useful.  This allows the producers to specify

> the lengths of the parent addresses that are regions in memory, and for

> consumers to determine them. The length should be allowed to be negative

> to handle the stack case cleanly and to indicate that it grows towards

> decreasing values (if length is negative the regions is from (address -

> offset) to address. For instance if I have an address in a physical

> object, a question that I might want to know is does this address

> overflow a containing memory region.

>

> 4) effectiveAddress can be one of two types of addresses: a relative

> address to some logical memory/file region, and an absolute memory

> address.  These concepts are independent, are both useful, and should be

> distinguishable from one another.  Something like:

>

> *offsetFromParent* - offset from containing parent (same as existing)

> *parentIndex*  - direct container address region if there is one (same

> as existing)

> *offsetFromLogicalAddressRegion*  - offset from some base logical region

> of memory (like effectiveAddress)

> *logicalAddressRegionIndex*  - the base logical region of memory the

> above offset if relative to

> *absoluteMemoryAddress* - absolute memory address of address (physical

> or virtual, be consistent).  Optional, recommended to only be included

> on addresses without parentIndex.

>

> Right now the effectiveAddress is difficult to use as they can not be

> compared without first figuring out what they are relative to and it

> isn't clear if the effective address is relative to something or absolute.

>

> Jim

>

> On 4/28/19 2:25 PM, Larry Golding (Myriad Consulting Inc) wrote:

>

> I created and pushed a change draft for Issue #401

> <https://nam06.safelinks.protection.outlook.com/?url="">>,

> âImproved address object designâ.

>

> https://nam06.safelinks.protection.outlook.com/?url="">

> <https://nam06.safelinks.protection.outlook.com/?url="">>

>

> The original design was ambiguous about what address.offset was based

> on: the parent object, or address.baseAddress?

>

>   * We renamed offset to offsetFromParent to clear up that ambiguity.

>   * We removed baseAddress because once itâs clear that âoffsetâ is

>     based on the parent object, thereâs no need to duplicate the

>     information in the child object.

>   * We added some more kind values ("header", "table") and we (really

>     Michael) made the definitions of "section" and "segment" more

>     precise. We (this time, really me 😊) also added text acknowledging

>     that the meanings of these terms are OS-dependent, and recommending

>     that SARIF producers use the term most appropriate for the target OS.

>   * Finally, an innovation: we introduced a property effectiveAddress,

>     which, when not present, can in certain circumstances be calculated

>     by adding the offsets all the way up the ancestor chain. (The

>     precise algorithm is in the spec.) This has two benefits:

>       o It allows the consumer to see the final address without having

>         to walk all the way up the ancestor chain. This at first /seems/

>         duplicative in the same way that baseAddress was (although even

>         if that were true, effectiveAddress saves the consumer more work

>         than baseAddress did because baseAddress duplicated information

>         in a /single/ address object rather than a whole chain of them).

>         *BUT!* Itâs not really duplicative, becauseâ

>       o The effectiveAddress property allows a consumer to see the final

>         address /even if some of the information required to calculate

>         it is missing from the ancestor addresses/. This allows you to

>         say, for example, âHere is the address of this entry in the

>         exception handler table; the table entry is a child of the

>         table, which is itself a child of some other thingâ â without

>         having to specify the address of the exception handler table if

>         you donât think thatâs relevant.

>

> This draft also includes a small amount of residual editorial feedback.

>

> As always, the provisional draft and its HTML version are up to date:

>

> https://nam06.safelinks.protection.outlook.com/?url="">

> <https://nam06.safelinks.protection.outlook.com/?url="">>

>

> https://nam06.safelinks.protection.outlook.com/?url="">

> <https://nam06.safelinks.protection.outlook.com/?url="">>

>

> *Please take a look.* We still plan to open the CSD 2 ballot on Monday.

>

> Thanks,

>

> Larry

>



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]