ISO/IEC JTC1/SC22/WG5 N2013 Result of the WG5 letter ballot on N2007 John Reid N2012 asked this question Please answer the following question "Is N2007 ready for forwarding to SC22 as the DTS?" in one of these ways. 1) Yes. 2) Yes, but I recommend the following changes. 3) No, for the following reasons. 4) Abstain. The numbers of answers in each category were: 4 for 1) Yes (Chen, Moene, Nagle, Whitlock) 2 for 2) Yes, but I recommend the following changes (Long, Reid) 4 for 3) No, for the following reasons (Bader, Cohen, Maclaren, Snyder) 2 for 4) Abstain (Corbett, Muxworthy) The ballot has failed - we have not reached consensus. Nevertheless, the comments are far less severe this time. I request J3 to prepare a revised version for approval by WG5 at the end of the June meeting. Here are the comments and reasons: Reinhold Bader 3) No, for the following reasons. The following items from my vote on the previous draft (see N1999) have not been dealt with yet: (5C), items (2),(3),(4),(7) (6B) Comments on section A.3.2.2 The conceptual changes in 14-111 (going from team depth to team variable specification for the purpose of intrinsic interfaces) have not been discussed in meeting 203. Furthermore, the following issues have come to my attention: Section 5: ~~~~~~~~~~ 5.1 I would welcome a NOTE at the end of [9:16-24] that clarifies how dummy arguments are treated: ---------- NOTE 5.1- A coarray dummy argument is not established in any ancestor team even if the corresponding actual argument is established in one or more of them. ---------- 5.3 Is it intended that a coarray-association as defined in [10:24-28] is permitted to use differing cobounds on each image? If not, should the cobounds be permitted to differ between teams? 5.4 In [11:11-12] determination of the image index is presumably based on the local cobounds even though those may be different from those in the sibling team (or specifically the targeted image). If the answer to the questions posed for 5.3 above is "NO", some additional restriction may be desirable for sibling team addressing, i.e. the data item must only be referenced via an associate name. If the answer to the first question is "YES", adding a note that describes the above situation is recommended. In any case, I consider it useful to extend the intrinsics IMAGE_INDEX and NUM_IMAGES to enable appropriate queries on sibling teams. 5.7 I suggest deleting the word "randomly" in the last line of NOTE 5.6. This is already implied by "chance of failure". Section 6 ~~~~~~~~~ 6.4 EVENT WAIT has the problem that it will probably hang forever if the image that is supposed to do the POST fails. While this can be ameliorated by (a) using EVENT_QUERY and FAILED_IMAGES to find out in advance whether this is the case (b) making (a) more reliable by dedicating a "monitor" image to investigate the above (a scenario where EVENT_QUERY on a remote event would be needed!) and do a bailout POST, I think adding some semantics to enable a properly defined exit status is desirable and allows for less klugey fail-safe programming. For example, one might have an additional optional argument ERROR_ON=[image-list|*] that causes the statement to exit with STAT=[STAT_STOPPED_IMAGE| STAT_FAILED_IMAGE] if something ill happens to an image in the specified image set. The LOCK, UNLOCK and possibly the SYNC MEMORY statements suffer from similar problems. "*" might refer to all images of the current team that have not failed or been stopped on entry to the statement. Section 7 Intrinsic procedures ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 7.4.[1-5] All atomic subroutines should permit optional STAT specification to permit signaling of a failure. I think it would be OK to specify that this may only work if the run-time can determine that the coindexed object that is subject of the atomic operation has become unavailable and after it has done so; otherwise the operation is expected to cause image failure itself. This is necessary to properly support fine-grained synchronization algorithms that can in principle continue even when image failures occur. (The example given by me in N1999/{Comments on section A.3.2.2} would hang if an image fails). To some extent, this issue is related to 6.4 above considering that EVENTs might be implemented in terms of atomics. 7.4.11 The text in [23:31-33] appears outdated given the recent changes to events. Also the procedure should be made atomic to make clear it can be invoked concurrently (i.e. in unordered segments) with updates being made via EVENT POST/WAIT. Furthermore, it should be made clear that an error can only arise if the queried event variable is on a failed image. Annex A: ~~~~~~~~ A.1.4 [42:18] Replace "p**q" by "p*q" A.2.1 I agree with previous comments that this example uses EVENT_QUERY incorrectly. An effort of improving the example also with respect to the fail-safe functionality is under way. _______________________________________________________________________ Malcolm Cohen No, the technical content is not yet agreed (see many comments by Nick Maclaren and Reinhold Bader), and I think it is not correct. In particular, (1) the collectives CO MAX, CO MIN, CO REDUCE, CO SUM, should be split into two forms, one with RESULT, one without. The one with RESULT should have SOURCE as INTENT(IN), the one without should have SOURCE as INTENT(INOUT). RESULT must not be optional. The SOURCE INTENT(IN) form should have no coarray restrictions on SOURCE. (2) the semantics of EVENT are still unclear. 6.3p3 says "If the segment that precedes an EVENT POST statement is unordered with respect to the segment that precedes another EVENT POST statement for the same event variable, the order of execution of the EVENT POST statements is processor dependent." but if they are in unordered segments, there is no order: processor- dependent or otherwise. (3) I agree that EVENT WAIT UNTIL_COUNT=-127 makes no sense, but it seems far more likely to be a progamming error than a deliberate intention to specify UNTIL_COUNT=1. There should be a requirement that the in the UNTIL_COUNT shall be positive so that processors can be encouraged to report such mistakes. (4) Note 6.2 should be normative text. Assuming that those are the semantics we want. (5) Perhaps Note 6.1 should be normative and state something like "The segment following execution of an EVENT POST statement is not ordered with respect to the segment following the corresponding EVENT WAIT statement.". Mind you that would require us properly describing the concept of "the corresponding EVENT WAIT" (which we seem to half-way have, but not clearly described). (6) Use of EVENT_QUERY + SYNC MEMORY looks very scary. It would be possible to use this to write remote-event-waiting, I am not entirely convinced that this is the right thing to do. Perhaps an EVENT POLL statement would be safer (or an option to EVENT WAIT e.g. BLOCK=.FALSE.). (7) And why doesn't EVENT QUERY simply say it returns the event count? If the event count can ever not be equal to "the number of successful posts minus the number of successful waits" there is a problem. If it is by definition equal to "the number of successful posts minus the number of successful waits", the wording is the problem. (8) Is the processor supposed to support the following: INTEGER,PARAMETER :: i4 = SELECTED_INT_KIND(18) EVENT WAIT(UNTIL_COUNT = HUGE(0_i4)) ??? That looks pretty gross so maybe not, but what about EVENT WAIT(UNTIL_COUNT = 100*1000*1000) ??? (And if that is still gross, EVENT_COUNT=1000 ...) For that matter, what about many EVENT POSTs before an EVENT WAIT. Is there a limit? Presumably yes, then let's (a) say there is a limit (b) require (or at least recommend) the processor to raise an error condition if the limit is exceeded (c) specify a minimum limit that the processor must implement; HUGE(0) springs to mind. _______________________________________________________________________ Bill Long 2) Yes, but I recommend the following change: At [36:9] change “5.6” to “5.8”. I agree with at least some of the editorial changes from other ballots, but will not reproduce them here. _______________________________________________________________________ Nick Maclaren 3) No, for the following reasons. Because this is a long response, the most serious and difficult points are: Events.2) This TS introduces a facility that is arguably in conflict with Fortran 2008 1.5 paragraph 1, and is not described as processor dependent. It is not clear how to fix it, and it should be removed. At the very least, it should be explicitly specified to be processor- dependent. Collectives.1) This TS introduces a conflict with Fortran 2008's concept and specification of variable definition context. This is definitely fixable, but may need significant changes to the collectives. General.1 and General.2) This TS should be explicitly removed from consideration for inclusion in the next Fortran standard (see N1979), because it will probably be be infeasible to specify a data consistency model by the deadline for the first CD ballot, and there is very unlikely to be sufficient implementation and user experience. EVENTS ------ Events.1) The wording in 8.7 page 32:26-27 still refers to EVENT POST and EVENT WAIT being within segments. 8.7 page 32:26-27 should replace: A coarray that is of type EVENT TYPE may be referenced or defined during the execution of a segment that is unordered relative to the execution of another segment in which that coarray of type EVENT TYPE is defined. by: A coarray that is of type EVENT TYPE may be referenced or defined during the execution of a segment that is unordered relative to the execution of the EVENT POST and EVENT WAIT statements by which that coarray is defined. Events.2) Reinhold Bader's (6B) [lines 258-264] and my Comment I [lines 923-965] in N1999 have still not been addressed. In addition, other comments along the same lines have been made in the past by other people. As a consequence, it is still completely unclear whether the statements in the example in 7.4.11 page 23:40-44 and 24:1-2 are actually correct. Indeed, in the absence of any progress guarantee, it is doubtful that example A.2.1 is permitted to always return zero from the call to EVENT_QUERY, thus going into an infinite loop irrespective of whether any events had been posted. If that is allowed, it would not have a defined interpretation, and would therefore not be a standard-conforming program (Fortran 2008 1.5 paragraph 1). See also Atomics.2 and General.1. Unless those questions can be resolved, and I do not see how, I remain of the opinion that EVENT_QUERY should be removed. If not, its effect and whether example A.2.1 will work at all should be explicitly stated to be processor dependent. TEAMS ----- Teams.1) The clarification that variables of type TEAM_TYPE have value semantics is welcome, but the TS also needs to state whether the values can be assigned between images and used on an image other than the one on which they were defined. It doesn't matter whether they can or cannot, but it should be explicitly stated which. Teams.2) It is strange to provide intrinsics like GET_TEAM and TEAM_ID yet provide no way to test whether two team values are the same. Equality and inequality comparison should be supported for TEAM_TYPE. After 5.2 page 9:30+ add: The module defines the following: The elemental operator == for two values of type TEAM_TYPE to return true if the values are the same and false otherwise. The elemental operator /= for two values of type TEAM_TYPE to return true if the values differ and false otherwise. Teams.3) The current wording for CHANGE TEAM still contains some implications of association semantics, which forbids VALUE and intrinsic copying. It has also been extended since N1996 to allow changing to the initial team, which is not in accordance with N1981, and I assume is an accident. 5.3 page 10:18-21 should replace: The shall have been defined by execution of a FORM TEAM statement in the team that executes the CHANGE TEAM statement or be the value of a team variable for the initial team. The values of the s on the imag es of the team shall be those defined by execution of the same FORM TEAM statement on all the images of the team. by: The values of shall have been established by execution of a FORM TEAM statement in the team that executes the CHANGE TEAM statement. The values of the s on all of the images of the team shall have been established by execution of the same FORM TEAM statement. Teams.5) The wording of SYNC TEAM seems to allow mixing values created by FORM TEAM with the initial ones, and is ambiguous about exactly which team it is collective over. It also refers to "the team variables for the initial team", which doesn't make sense as there will usually be no such variables. 5.6 page 12:21-25 should replace: The SYNC TEAM statement is an image control statement. The value of shall have been established by execution of a FORM TEAM statement by the current team or an ancestor of the current team, or be the value of a team variable for the initial team. The values of the s on the images of the team shall be those defined by execution of the same FORM TEAM statement on all the images of the team or shall be the values of the team variables for the initial team. by: The SYNC TEAM statement is an image control statement. The value of shall have been established by execution of a FORM TEAM statement by the current team or an ancestor of the current team, or shall specify the initial team. The values of the s on the images of the team specified by the shall all specify the same team. Teams.6) The example in 7.4.15 page 26:8-19 still asserts more than is in the normative text. The whole reason for adding NEW_INDEX to FORM TEAM was to allow control over the image ordering but leave the default ordering processor dependent. This is not a mere nitpick - examples like this are a major cause of erroneous interpretations by programmers. 7.4.15 page 26:15 should be replaced by: : ! Code for half of the images in the current team 7.4.15 page 26:17 should be replaced by: : ! Code for the other half of the images in the current team Teams.7) 5.5 page 12:7 refers to "the same team", but that is not wholly clear. A Note should be sufficient. See below. Teams.8) There is still nothing said about when resources may be released, which is a sure recipe for some programmers assuming that they will be and some implementors not doing it. A simple Note would be better than nothing. The following Note addresses both points. After 5.5 page 12:18+, append: NOTE Each execution of the FORM TEAM statement establishes a new team for each value of . Separate executions or separate statements establish separate teams, even if the images and values of are identical. Programmers should not assume that FORM TEAM can be called an indefinite number of times, though processors are encouraged to support that. ATOMICS ------- Atomics.1) 7.2 and A.3.2 are truly massive improvements and, as far as I can see, make the intent, requirements and non-requirements clear. However, I think that there should be a paragraph saying explicitly that the matter of progress is not addressed by the TS, and is processor dependent. A.3.2.2 page 46:9+, add: This includes the concept often called 'progress'. A change to an atomic variable in a segment may become visible to another image 'immediately', or may not until that image reaches a segment that is ordered after the segment in which it was changed. Atomics.2) In 7.2 page 17:20-21, the TS says that developing a formal data consistency model is left until integration into the main standard, but it is important to note that such a model is NOT a matter of the atomics alone. It will be much harder to specify the interactions between the atomics, collectives, events and the existing execution order model, as well as when a processor is required to make progress (if ever). See also Event.2 and General.1. Collectives ----------- Collectives.1) This is an old objection, where I was mollified by people saying that intrinsics need not follow the same rules as ordinary procedures. However, on thinking it over, that is not an adequate answer. Variable definition context is strictly defined (i.e. not processor dependent) in Fortran 2008 16.6.7 and elsewhere, and some constraints (especially C539) require a compile-time diagnostic to be issued if a read-only entity is used in one. The question is when SOURCE constitutes a variable definition context. N2007 does not say so, and it is not one of the items listed in 16.6.7. a) If it always is, then collectives cannot be used on INTENT(IN) arguments altogether, which is very poor software engineering. b) If it never is, then we have defined a circumstance under which a variable can become defined that is not a variable definition context, which is worse. c) It cannot be dependent on whether it can be modified, as that is not statically determinable, though that is all that can be deduced from the current wording. d) It could be dependent on the presence of a RESULT argument. That introduces a new syntactic concept into the base standard, which is dubiously within the scope of N1981, but is feasible. Collectives.2) The specification introduces a 'gotcha' where code like the following will cause data corruption: SUBROUTINE Collective (array, target_image) INTEGER :: array, target_image CALL CO_MAX(array,target_image) END SUBROUTINE Collective While that is the user's mistake, the standard prevents any kind of diagnostic, which is poor software engineering. Collectives.3) They are unnecessarily restrictive, in that there is no good reason that the result should not be obtainable on an arbitrary subset of images. Collectives.4) While I can see why CO_BROADCAST is exceptional in having solely an in-place form, there are circumstances under which a copying form is wanted. I agree that a foolish consistency is the hobgoblin of little minds, but this is a case where there are good reasons to be consistent, even if the extended function is rarely needed. If this specification is preserved, at least the first issue should be resolved in normative text, and preferably the others as well. As an example of what I regard as a better approach, here is a rewrite of CO_MAX. The replicated wording for SOURCE and OBJECT could easily be condensed, but I have not done so here. 7.4.7 CO_MAX (SOURCE [, RESULT, RESULT_IMAGE, STAT, ERRMSG]) or CO_MAX_IN_PLACE (OBJECT [, RESULT_IMAGE, STAT, ERRMSG]) Description. Compute elemental maximum value on the current team of images. Class. Collective subroutine. Arguments. SOURCE shall be of type integer, real, or character. It is an INTENT(IN) argument. It shall have the same type and type parameters on all images of the current team. If it is a scalar, the computed value is equal to the maximum value of SOURCE on all images of the current team. If it is an array it shall have the same shape on all images of the current team and each element of the computed value is equal to the maximum value of all the corresponding elements of SOURCE on the images of the current team. RESULT (optional) shall be of the same type, type parameters, and shape as SOURCE. It is an INTENT(OUT) argument. OBJECT shall be of type integer, real, or character. It is an INTENT(INOUT) argument. It shall have the same type and type parameters on all images of the current team. If it is a scalar, the computed value is equal to the maximum value of OBJECT on all images of the current team. If it is an array it shall have the same shape on all images of the current team and each element of the computed value is equal to the maximum value of all the corresponding elements of OBJECT on the images of the current team. [[[ Unchanged RESULT_IMAGE (optional) shall be a scalar of type integer. It is an INTENT(IN) argument. If it is present, it shall be present on all images of the current team, have the same value on all images of the current team, and that value shall be the image index of an image of the current team. STAT (optional) shall be a scalar of type default integer. It is an INTENT(OUT) argument. ERRMSG (optional) shall be a scalar of type default character. It is an INTENT(INOUT) argument. ]]] For the purpose of matching the sequence of invocations of collective subroutines [7.3 paragraph 1], CO_MAX and CO_MAX_IN_PLACE are the same collective. If RESULT_IMAGE is present, either OBJECT or RESULT shall be present on the image specified by RESULT_IMAGE, and the computed value is assigned to that argument. If RESULT_IMAGE is not present, the computed value is assigned to OBJECT or RESULT on all the images of the current team that have either argument present. [ Issues: 1) Would it be better to say that any argument associated with OBJECT or RESULT on other images than one specified by RESULT_IMAGE becomes undefined? Or not used? Or processor dependent? Or simply unspecified, as in N2007? I have no strong view. 2) If no image has either OBJECT or RESULT, this is a waste of time, but is perfectly well-defined. That could easily be forbidden, though I can't see why it need be. ] [[[ Unchanged The effect of the presence of the optional arguments STAT and ERRMSG is described in 7.3. Example. If the number of images in the current team is two and SOURCE is the array [1, 5, 3] on one image and [4, 1, 6] on the other image, the value of RESULT after executing the statement CALL CO_MAX(SOURCE, RESULT) is [4, 5, 6] on both images. ]]] GENERAL ------- General.1) I remain extremely concerned about potential inconsistencies and unimplementabilities in the specification, especially caused by interactions between features. In particular, several aspects introduced since N1958/N1863 are beyond the domain of established computer science - and, as far as I know, support for 'process' failure has never been successfully standardised before, despite several attempts over the past half-century. The point here is not that this TS should be delayed until it has a data consistency model, but that we cannot be certain that one is even definable. Half a century of experience in this area has taught us that intuition is not a reliable guide, and 'obvious truths' are often false when parallel data consistency is involved. As an example in a much more widely-used language than Fortran, Java has had threading support and a memory model since about 1995 but it was shown, only in 2004, to be badly flawed; it was replaced in 2005. http://www.ibm.com/developerworks/library/j-jtp02244/index.html In the case of the facilities in this TS, I cannot either produce examples of deadlock or causal loops, or convince myself that they do not exist. However, I can convince myself that livelock and infinite loops (see Events.2), in the case where the program's logic contains no such flaws, are possible with the current specification of EVENT_QUERY. I do not believe that developing a proper data consistency model, formal or informal, will be possible by February 2015. As mentioned in Atomics.2 and Events.2, this is NOT a matter of the atomics alone, but of the interactions between the atomics, collectives, events and the existing execution order model, as well as when a processor is required to make progress (if ever). A proposal is made below. General.2) I also remain extremely concerned about whether this specification will be reliably and efficiently implementable, for a sufficiently high proportion of user programs, except on systems where the hardware or operating system has special support for it. I accept that people who understand the implementation issues will be able to write portably efficient programs, and SOME users will do so, but the the question is how many. A specific issue here is that Fortran is maintaining its position largely because it can be used to write more efficient code than its competitors. If, in the eyes of the user community, it loses that edge, it will disappear. In particular, I cannot see how to to implement it portably, reliably and efficiently, using only the reliably portable facilities of POSIX, MPI and TCP/IP. These do not include MPI passive one-sided communication or even MPI_THREAD_MULTIPLE, largely because there is no one-sided synchronisation in POSIX. My understanding is that there are, as yet, only two full and released implementations of Fortran 2008 coarrays (Cray's and Intel's), and very little user experience with the latter; IBM's may be imminent, but I know of no others. In all cases, the implementation works only with the vendor's own MPI, as far as I know. It seems very unlikely indeed that there will be significant experience with implementing and using the facilities in this TS before February 2015. A proposal is made below. General.3) This comment is made mainly because it supports the previous points in this section. It is dubious that this TS meets requirement S1 in N1981 any longer, as it is over double the length as and much more complex than the original proposal (N1858 and N1863). Events can be regarded as replacing NOTIFY/QUERY and parallel I/O has been dropped, but major new, interacting, features include the new atomics, non-hierarchical team usage, and support for image failure. As a consequence of the above comments, I believe that this TS should be explicitly removed from consideration for inclusion in the next Fortran standard (see N1979), because the first CD ballot of the next revision of the standard is due in February 2015. _______________________________________________________________________ David Muxworthy I vote abstain with two minor editorial comments: 1. Corrigendum 1 should also be listed on page 3 because Fortran corrigenda are not cumulative. Corrigendum 3 will have almost certainly have been approved by the time this WG5 ballot is completed (SC22 ballot deadline today, 2014-04-15) and so should also be listed. 2. Technical Corrigendum 1 changed the heading of 13.7.165 from THIS IMAGE ( ) or THIS IMAGE (COARRAY [, DIM]) to THIS IMAGE ( ), THIS_IMAGE (COARRAY) or THIS_IMAGE (COARRAY, DIM) The edits at 35:27-29, 35:32 should reflect this. ______________________________________________________________________ John Reid 2) Yes, but I recommend the following changes. 1. Failed images There has not been adequate discussion of an image stalling because of the failure of another image, which will cause it to fail despite there being nothing wrong with its hardware. All we have is a warning in NOTE 5.7. If restarting is planned, as in the example A.1.2, such images should be available for reuse. Such stalling anyway indicates that the team calculation has gone wrong, so I suggest addding a RENDEZVOUS specifier to the SYNC ALL statement. If a stalled image is detected, all images of the team would re-execute the most recently executed RENDEZVOUS FOR ALL statement with STAT= value set to STAT_FAILED_IMAGE, ignoring all pending synchronizations initiated since that FOR ALL was last executed. Alternatively, all images of the team could exit the change team construct. That would give the required effect in A.1.2 without any code change. The programmer who wishes to guard against stalling because of accessing an image that has failed may do do as follows: IF(ALL(FAILED_IMAGES()/=i)) THEN a = a[i] ELSE ! Do something else END IF This is illustrated in my rewrite of A.2.1 below. I think we may need a logical function that tests a given image for failure. 2. Teams I am unhappy with the idea a code using an image selector for ancestor team not visible in the scope. For example, a library code should leave such work to the calling code. We have chosen to address an ancestor in an image selector by using its team variable name. We should be doing the same for other references to an ancestor. An ancestor team may be at different team distances on executing images of different descendant teams. I suggest the replacement of DISTANCE by TEAM as arguments of TEAM_ID, NUM_IMAGES, and THIS_IMAGE, the removal of DISTANCE as an argument of GET_TEAM, and the removal of TEAM_DEPTH. 3. Edits [10:27] Change "Apart from its final upper bound, its" to "Its". [There is nothing special about the codimension-decl here, so there is no need to say anything about the final codimension.] [10:30] Change "established." to "established, apart from its final upper cobound". [Here, the final upper cobound is likely to be different.] [13:10-13] Replace by "The value of the default integer scalar constant STAT_FAILED_IMAGE is positive and different from the value of STAT_STOPPED_IMAGE, STAT_LOCKED, STAT_LOCKED_OTHER_IMAGE, or STAT_UNLOCKED. If the processor detects that an image of the current team has failed, the". [With the addition of FAIL IMAGE, we want a processor that cannot detect a truly failed image to respond to the execution of FAIL IMAGE.] [15:6-24] Should an event variable be atomic? [15:34] Wording like that at [17:12-14] is needed here. [23:26] EVENT_QUERY should be an atomic subroutine. [40:4, 40:25-32] To make the code tolerate failing spare images, replace the DO loop by the following: k = images_used DO i = 1, size(failed_img) IF (failed_img(i) == 1) ERROR STOP 'cannot recover' DO k = k+1, num_images() IF (all(failed_img(:)/=k) EXIT END DO IF (me == k) THEN me = failed_img(i) id = 1 EXIT END IF END DO If this is done, [40:4] should be deleted. [43:6-44:34] This example has bugs. I think the message would be clearer if this example were replaced by an example that is not tolerant to failed images, followed by a modification that is. Here is a draft. A.2.1 EVENT_QUERY example The following example illustrates the use of events via a program in which image 1 acts as master and shares out work items to the other images. Only one work item at a time can be active on a worker image, and each deals with the result (e.g. via I/O) without directly feeding data back to the master image. Because the work items are not expected to be balanced, the master keeps cycling through all the images to find one that is waiting for work. An event is posted by each worker to indicate that it has completed its work item. Since the corresponding variables are needed only on the master, we place them in an allocatable array component of a coarray. An event on each worker is needed for the master to post the fact that it has made a work item available for it. PROGRAM work_share USE, INTRINSIC :: iso_fortran_env USE :: mod_work, ONLY: & ! Module that creates work items work, & ! Type for holding a work item create_work_item, & ! Function that creates work item process_item, & ! Function that processes an item work_done ! Logical function that returns true if all work done TYPE(event_type) :: submit[*] ! Whether work ready for a worker TYPE :: asymmetric_event TYPE(event_type), ALLOCATABLE :: event(:) END TYPE TYPE(asymmetric_event) :: free[*] ! Whether worker is free TYPE(work) :: work_item[*] ! Holds all the data for a work item INTEGER :: count, i, n, nbusy[*] IF (this_image() == 1) THEN ! Get started ALLOCATE(free%event(2:num_images())) nbusy = 0 ! This holds the number of workers working DO i = 2, num_images() ! Start the workers working IF (work_done()) EXIT nbusy = nbusy + 1 work_item[i] = create_work_item() EVENT POST (submit[i]) END DO ! Main work distribution loop master : DO image : DO i = 2, num_images() CALL EVENT_QUERY(free%event(i), count) IF (count == 0) CYCLE ! Worker is not free EVENT WAIT (free%event(i)); nbusy = nbusy - 1 IF (work_done()) CYCLE nbusy = nbusy + 1 work_item[i] = create_work_item() EVENT POST (submit[i]) END DO image IF ( nbusy==0 ) THEN ! All done. Exit on all images. DO i = 2, num_images() EVENT POST (submit[i]) END DO EXIT master END IF END DO master ELSE ! Work processing loop worker : DO EVENT WAIT (submit) IF (nbusy[1] == 0) EXIT CALL process_item(work_item) EVENT POST (free[1]%event(this_image())) END DO worker END IF END PROGRAM work_share A.2.1a EVENT_QUERY example that tolerates image failure This example is an adaptation of the example of A.2.1 to make it able to execute in the presence of the failure of one or more of the worker images. The function create_work_item now accepts an integer argument to indicate which work item is required. It is assumed that the work items are indexed 1, 2, ... . It is also assumed that if an image fails while processing a work item, that work item can subsequently be processed by another image. The internal subroutine failed tests whether a particular image has failed. PROGRAM work_share USE, INTRINSIC :: iso_fortran_env USE :: mod_work, ONLY: & ! Module that creates work items work, & ! Type for holding a work item create_work_item, & ! Function that creates work item process_item, & ! Function that processes an item work_done ! Logical function that returns true if all work done TYPE(event_type) :: submit[*] ! Whether work ready for a worker TYPE :: asymmetric_event TYPE(event_type), ALLOCATABLE :: event(:) END TYPE TYPE(asymmetric_event) :: free[*] ! Whether worker is free TYPE(work) :: work_item[*] ! Holds all the data for a work item INTEGER :: count, i, k, kk, n, nbusy[*], np, status INTEGER, ALLOCATABLE :: working(:) ! Items being worked on INTEGER, ALLOCATABLE :: pending(:) ! Items pending after image failure IF (this_image() == 1) THEN ! Get started ALLOCATE(free%event(2:num_images())) ALLOCATE(working(2:num_images()), pending(num_images()-1)) nbusy = 0 ! This holds the number of workers working k = 1 ! Index of next work item np = 0 ! Number of work items in array pending DO i = 2, num_images() ! Start the workers working IF (work_done()) EXIT work_item[i] = create_work_item(k) working(i) = k k = k + 1 nbusy = nbusy + 1 EVENT POST (submit[i], STAT=status) IF (status==STAT_FAILED_IMAGE) THEN working(i) = 0 k = k - 1 nbusy = nbusy - 1 END IF END DO ! Main work distribution loop master : DO image : DO i = 2, num_images() IF (ANY(FAILED_IMAGES()==i)) THEN ! Image has failed IF (working(i)>0) THEN ! It failed while working np = np + 1 pending(np) = working(i) working(i) = 0 END IF CYCLE image END IF CALL EVENT_QUERY(free%event(i), count) IF (count == 0) CYCLE image ! Worker is not free EVENT WAIT (free%event(i)) nbusy = nbusy - 1 IF (np>0) THEN kk = pending(np) np = np - 1 ELSE IF (work_done()) CYCLE image kk = k k = k + 1 END IF nbusy = nbusy + 1 working(i) = kk work_item[i] = create_work_item(kk) EVENT POST (submit[i],STAT=status) ! If image i has failed, this will not hang and the failure ! will be handled on the next iteration of the loop END DO image IF ( nbusy==0 ) THEN ! All done. Exit on all images. DO i = 2, num_images() EVENT POST (submit[i],STAT=status) IF (status==STAT_FAILED_IMAGE) CYCLE END DO EXIT master END IF END DO master ELSE ! Work processing loop worker : DO EVENT WAIT (submit) IF (nbusy[1] == 0) EXIT worker CALL process_item(work_item) EVENT POST (free[1]%event(this_image())) END DO worker END IF END PROGRAM work_share _______________________________________________________________________ Van Snyder Comments fall into three categories: 1. Stuff that might be profitably added. 2. Stuff I would do differently. 3. Other stuff Stuff that might be profitably added ==================================== It is processor dependent whether common or independent random number generators are used, and there's no way to detect which is the situation. Therefore taking action using one assumption or the other (e.g., calling or not calling RANDOM_SEED on each image with a hopefully-different value, maybe dependent on the image number) might be the wrong thing to do. It is preferable to provide a named constant in ISO_FORTRAN_ENV to indicate whether the processor uses a common random number generator on all images, or intrinsic procedures so to inquire or specify. References to the intrinsic subroutine that specifies whether common or independent random number generators are used should be image control statements. 13.5p4 should be adjusted so that the segment ordering requirement applies only if a common random number generator is used. Note 5.8 suggests that there ought to be a mechanism to specify, change, and inquire the image to which standard input is connected. The OPEN and INQUIRE statements come immediately to mind. The image index should be in the initial team, and connecting should, for consistency, be allowed in only one image. Either that, or standard input should be connected to all images. A READ statement for a file that is attached to more than one image should be specified to execute as if in a critical section. A pointer can have a coarray target, but cannot be directly coindexed, although if they're actual arguments associated with (necessarily nonpointer, so far) coarray dummy arguments, their target can be coindexed in the invoked procedure. Allow coarrays to be pointers. Prohibit allocating and deallocating coarray pointers. In pointer assignment, require that if the pointer is a coarray, the data target shall be a coarray. Stuff I would do differently ============================ [12:6] It's a bit easier to compute team numbers if zero is allowed. E.g., mod(this_image,4). Replace "be greater than zero" with "not be negative" and replace "1" with "zero" at [26:6] [15:14-15] It would be better if variables of EVENT_TYPE were not required to be coarrays. Instead eliminate C601 and constrain to be a coarray, a coindexed object, or a pointer in C604, and a coarray or a pointer in C605. In pointer assignment, if a is of type EVENT_TYPE, or of a type that has a noncoarray potential subobject component of type EVENT_TYPE, constrain the to be a coarray or a pointer. Thereby, by induction, a EVENT_TYPE pointer can only be associated with a coarray. 14-138 advocates the same for LOCK_TYPE. [15:16-24 C602, C603] Protected types would be a better solution. If EVENT_TYPE were a protected type, or by adding "C602a A subobject of an event variable shall not appear in a variable definition context, shall not be an actual argument in a reference to a procedure with implicit interface, and shall not be an actual argument associated with a dummy argument that has unspecified intent." at [15:24+], it would be possible to expose the event count as a specific component name, and eliminate EVENT_QUERY. Since we have decided that we do not want to allow type-bound procedures to be actual arguments or procedure pointer targets, we could allow to invoke type-bound functions that have no arguments without parentheses enclosing the (empty) argument list. A corresponding change should allow to reference a scalar with an empty subscript list. Thereby, it would be impossible to know, and impossible for the syntax to depend upon, whether event%count or event%count() is a reference to a scalar component or a type-bound function that has no arguments. This is intended as food for future thought; it's outside the scope of the present TS. If protected types existed, LOCK_TYPE should also be a protected type. [17:24-25] Since references to collective subroutines have to be invoked by the same statement on all nonfailed images of the current team, and it's hard to imagine how they can produce an output argument value that's the result of a collective action until all of them finish, it would be simpler either to make references to them be image control statements (and establish segment boundaries), or to specify that executing them includes the effect of executing an image control statement. (This is done by the edit proposed for [32:21+] below.) This would also affect the discussion at [47:15-16]. [18:11ff General for 7.4.2, 7.4.4 and 7.4.5] Since intrinsic procedures are "magic" it would be possible (and more economical) to replace ATOMIC_AND, ATOMIC_OR and ATOMIC_XOR with ATOMIC_OP ( ATOM, OPERATOR, VALUE ) or ATOMIC_OP ( ATOM, OPERATOR, VALUE, OLD ), and require OPERATOR to be a local name for one of the intrinsic functions IAND, IEOR or IOR. Yes, this is an exception to the general rule that generic names cannot be actual arguments, but so what? Intrinsics are magic. [18:30ff General for 7.4.7, 7.4.8] Since intrinsic procedures are "magic" it would be possible (and more economical) to delete CO_MAX and CO_MIN in favor of CO_REDUCE, and allow the OPERATOR of CO_REDUCE to be a local name for one of the intrinsic functions MAX or MIN. It would be nice to include PRODUCT and SUM under this rubric, but they don't have two arguments, so explanation of CO_REDUCE might become difficult. Yes, this is an exception to the general rule that generic names cannot be actual arguments, but so what? Intrinsics are magic. [24:20-25:20] Since TEAM_TYPE is not prohibited in a variable definition context, GET_TEAM could be a function instead of a subroutine. This would be handy for passing team variables as actual arguments. [32:21+] Insert "{In 8.5.1 Image control statements, insert a list item} 'a CALL statement that references an intrinsic collective subroutine'". Other stuff =========== [General] Turn on paragraph numbering. [General] Process with LaTeX such that ligatures are not used. This makes words that contain, e.g., "fi", searchable in Acrobat Reader. [General] Use either "in the current team" or "of the current team", and either "in the initial team" or "of the initial team", throughout, not one or the other at random. Using "in" might make more sense since "image of the current team" need not necessarily be read as "image that is a member of the current team." It might be read as if there is, somehow, an "image" (with a meaning different from the one in Subclause 2.3.4 of 1539-1, e.g., "a photograph", or a copy -- "image" -- of the team variable), that reflects or represents all of the current team. [General] "positive" appears in some places, while "greater than zero" appears in other places. Use "positive" throughout (because it's shorter). [General] "less than or equal to" appears in some places, while "not greater than" appears in other places. Use "not greater than" throughout (because it's shorter). [Foreword iii:p5:line 3] Is "must" the correct word here? [Introduction iv:p1:2] Delete "a set of". [Introduction iv:p2:6] Delete "for". [Introduction iv:p2:8] Replace "progress" with "proceed" because "progress" does not appear anywhere else, except in notes, in either the TS or 14-007. Either that, or define "progress" (as a verb) in the "Terms and definitions" subclause. [Introduction iv:p3:5] Delete "sets of". [9:26-30] Should TEAM_TYPE be specified to have a public type-bound defined assignment? Or does it not matter because one cannot tell the difference? [10:7 C503] Replace "A" with "An". [10:13 C505] After "" insert ", or the same as a in another ,". [10:31-34] Raises the question whether allocatable non-coarray variables that are not allocated at the beginning of a CHANGE TEAM construct are deallocated at the end. Probably not. If yes, it should be specified. If not, it should be mentioned in a note. [10:36] The phrase "team containing the executing image that is identified by " should almost certainly be "current team". I don't see how things can be otherwise. If some subset of the current team decide they're going to be in one subteam, and only those images execute the CHANGE TEAM statement, from the point of view of other images of the (previously) current team, the images of that subteam are still part of their team; the result is overlapping teams, until other images, that are members of other teams, execute a different CHANGE TEAM statement, maybe in an unordered segment. I thought the intent of the CHANGE TEAM construct was to partition the current team, so that no image can ever be a member of more than one currently-executing team. In any case, the phrase "executing image that is identified by " appears to be nonsense. A identifies a set of images, not one image. I don't see a clean way to reorganize the sentence to make it clear that " applies to "the team", not "the executing image". [12:6-7] Replace "be greater than zero and is the same for all images that are members of the same team" with "be positive. Images are members of the same team if and only if they specify the same value of ." (I prefer "not be negative" instead of "be positive". It's a bit easier to compute team numbers if zero is allowed. See the remark for [12:6] in the "Stuff I would do differently" section above.) [12:12] Replace "assigned by the processor" with "a processor dependent value that shall be positive and not greater than the number of images in the team". [13:Note 5.6] There are assumptions about RANDOM_NUMBER here. If all images produce exactly the same sequence of random numbers, they will all fail, pretty much at the same time. Maybe insert something to the effect "if the random number generators on different images are independent and are started with seeds that produce different sequences, or a common random number generator is used...." [13:20] Replace "defined" with "declared". [14:Note 5.8] Replace "image 1" with "image 1 in the initial team" twice. [15:7] Append "It does not have the BIND attribute, and is not a sequence type." [15:9-10] Replace "The effect of each change ... change" with "The event count is updated as if by invoking the ATOMIC_ADD intrinsic subroutine" to avoid any suspicion that a new kind of mechanism is in play here. [15:16] Replace "An" with "A nonpointer". [15:19,24] Replace "where" with "if" twice. [15:21] Insert "nonpointer" before "variable". {It would be better to append "unless both the actual argument and dummy argument are pointers" at the end of 16.6.7p1(12), but that ought to be done in the normal course of a revision, not sneaked in via a TS.} [15:24+] Insert a constraint "C603a (R425) If EXTENDS appears and the type defined has a potential subobject component of type EVENT_TYPE from the intrinsic module ISO_FORTRAN_ENV, its parent type either shall be EVENT_TYPE or shall have a potential subobject component of type EVENT_TYPE." This is actually undesirable to put in the standard. What we really want to do is to combine the protection of EVENT_TYPE and LOCK_TYPE from by polymorphism, so that one could extend EVENT_TYPE and give a component of LOCK_TYPE and vice-versa; therefore to re-write C435 to say C435 (R425) If EXTENDS appears and the type defined has a potential subobject component of type EVENT_TYPE or LOCK_TYPE, from the intrinsic module ISO_FORTRAN_ENV, its parent type either shall be EVENT_TYPE or LOCK_TYPE, or shall have a potential subobject component of type EVENT_TYPE or LOCK_TYPE. [15:24+] Insert a paragraph "The value of the actual argument corresponding to the CPTR argument of the C_F_POINTER subroutine from the ISO_C_BINDING intrinsic module shall not be the address of an object of type EVENT_TYPE, unless the FPTR argument is a pointer of the same type as the object." [15:24+] Note 13.29 in 14-007, concerning lock variables, was apparently considered to be useful. Insert a similar note concerning event variables: "Note 6.0 The restrictions against changing an event variable except via EVENT POST and EVENT WAIT statements ensure the integrity of its value and facilitate efficient implementation, particularly when special synchronization is needed for correct event handling." [15:26, 30-31] Replace the introductory paragraph with "Successful execution of an EVENT POST statement posts an event, consisting of atomically incrementing the count of the event variable by 1. If an error condition occurs during execution of an EVENT POST statement, the count does not change. The EVENT POST statement is an image control statement." Then delete the paragraph immediately after C604. Using "atomically" here is important. Either that, or it is necessary to say that an EVENT POST statement executes as if in a critical section. [15:29] Delete "the". [16:2] Replace the introductory paragraph with "Successful execution of an EVENT WAIT statement waits until a specified number of events are posted by successful execution of EVENT POST statements. The EVENT WAIT statement is an image control statement." [16:8] Replace "threshold of its event argument" with "threshold value". Otherwise, "threshold value" in the next two steps is not a defined concept. Don't use "argument" since the EVENT WAIT statement is not a procedure reference. [16:12] Replace "event count" with "count of the event variable", for consistency with [15:30] and [16:10]. [17:32] Delete first "the". [17:33] Replace "on" with "in". [17:38] Insert "processor-dependent" before "nonzero". Should there be a requirement that the nonzero value be different from STAT_STOPPED_IMAGE or STAT_FAILED_IMAGE? [18:1] Insert "the named constant" before "STAT_STOPPED_IMAGE" and replace "in" with "from". [18:3] Insert "the named constant" before "STAT_FAILED_IMAGE" and replace "in" with "from". [18:4] Replace "as" with "to be" or "to have". [18:10+ Note 7.1] I can't figure out the point of the note. If it isn't pointless, replace "undefined ... collective with RESULT" with "undefined if an error condition occurs during execution of a collective subroutine and the RESULT argument is". [18:16 19:2] Delete "and" twice. [19:9-11] Can we put the argument value conditions first? For example "CALL ATOMIC_AND ( I[3], 6, IOLD ) with the value of I[3] equal to 5 causes...." [19:16 19:34] Delete "and" twice. [19:41-43] Can we put the argument value conditions first? [20:5] Delete "and". [20:1 20:12 34:14+9] Perhaps ATOMIC_XOR should be ATOMIC_EOR since the intrinsic function it says it uses "under the covers" is IEOR. [20:7] Repair the typesetting so the line doesn't end with left parenthesis. [20:12-14] Can we put the argument value conditions first? [20:19-22] The SOURCE argument to CO_REDUCE is prohibited to be polymorphic. Can the SOURCE argument to CO_BROADCAST be polymorphic? If so, insert "dynamic" before "type" and replace "parameters" with "parameter values" (the parameters are necessarily the same on all images, because all images execute the same statement). If not, replace the sentence with "shall be a scalar or array of any type. It shall not be polymorphic. Each length parameter shall have the same value on all images of the current team." The first paragraph of 7.3 requires the subroutine to be invoked by the same statement on all images of the current team, and therefore the declared type and kind type parameter values cannot be different. [19:22] Append "It is an INTENT(INOUT) argument." The associated actual argument cannot be an expression because all images execute the same statement, and a new value is assigned on all but SOURCE_IMAGE. This cannot be done for CO_MAX etc., for reasons already explained by Nick. [20:34-35] Delete "It shall have... team." because it cannot be otherwise. [20:40,41] Either replace "RESULT" with "it" at line 40, or replace "It" with "RESULT_IMAGE" at line 41. [21:5] Delete "the" before "images". [21:7, 21:34, 23:15] Is it really necessary for SOURCE to become undefined on images other than RESULT_IMAGE if RESULT is not present? Why not simply "unchanged"? [21:20-21] Delete "It shall have... team." because it cannot be otherwise. [21:26-27] Either replace "RESULT" with "it" at line 26, or replace "It" with "RESULT_IMAGE" at line 27. [21:32] Delete "the" before "images". [22:1-2] ER-RMSG is strange hyphenation. If it doesn't fit if hyphenated ERR-MSG, put it all on the next line. [22:6-7] Delete "It shall have... team." because it cannot be otherwise. [22:17,19] Either replace "RESULT" with "it" at line 17, or replace "It" with "RESULT_IMAGE" at line 19. [22:44-23:1] Delete "It shall have... team." because it cannot be otherwise. [23:7-8] Either replace "RESULT" with "it" at line 7, or replace "It" with "RESULT_IMAGE" at line 8. [23:13] Delete "the" before "images". [23:24-24:1] Since EVENT_QUERY almost certainly internally consists only of looking at a component of a coindexed object, why does it have STAT and ERRMSG arguments? A reference to a coindexed object in, say, an expression, doesn't have them. It would be simpler if EVENT_QUERY were a function, maybe better as a type-bound function than an intrinsic function. Even better would be if EVENT_TYPE were a protected type and COUNT were a public component. [23:30] Replace "no" with "not". [23:32] Subtracting one for each successfully-executed EVENT WAIT statement doesn't account for the UNTIL_COUNT specifier in the EVENT WAIT statement. But why specify the computation here (again)? Replace "number of successful posts minus the number of successful waits" with "the event count of the EVENT argument". [23:35] Replace "defined" with "dependent". [25:36] Is it possible for EVENT_QUERY to set STAT to STAT_STOPPED_IMAGE or STAT_FAILED_IMAGE? Even if not, should there be a requirement that the error status be different from those values (at least to future-proof it)? [24:10] Insert "decimal" before "range". Replace "be at least as large as" with "not be less than that" (for consistency with [23:30]). [26:6] Replace "1" with "zero" if it is decided that team numbers can begin at zero (see the edit above for [12:6-7] and the remark for [12:6] in the "Stuff I would do differently" section above). [26:40] Insert "processor-dependent" before "nonzero". Should there be a requirement that the nonzero value be different from STAT_STOPPED_IMAGE or STAT_FAILED_IMAGE (at least to future-proof it)? [27:12,14] After "present" insert "or is present with the value false", then delete ", otherwise ... specified". [29:9] Replace "a" with "as". [29:16] Replace "it" with "the subset" to avoid confusion with "it" meaning "the program". [29:21] Insert "for" before "performing". [29:23] Replace "features for the support of" with "facilities to support". [29:24] Replace "features" with "facilities". [30:11] After "team" insert ", consisting of all images," [31:2] Is it necessary or useful to mention access to images of other teams here? [31:13] Insert a comma after "appears". [33:35-32:5] Are these the only possible nonzero values? If not, add a sentence about any other errors resulting in a processor-dependent positive value different from STAT_.... [31:37] Replace "in" with "from" for consistency with 6.7.4 etc. [32:1] Replace "in" with "from". [32:4] Replace "in" with "from". [32:5] Insert "allocation" before "status". [32:36] Delete "that is". [33:17-25] Are these the only possible nonzero values? If not, add a sentence about any other errors resulting in a processor-dependent positive value different from STAT_.... [33:20,21,24] Replace "in the intrinsic" with "from the intrinsic" thrice. [33:37+] Insert a new subclause "8.7a Edits to Clause 9 {In subclause 9.5.1, paragraph 4, concerning an asterisk in a READ statement, insert 'in the initial team' after 'image 1'.}" [34:7+] Insert "{In Note 13.1 replace 'atomic variables' with 'access to variables using atomic subroutines'}" because Fortran does not have atomic variables, and the term is not used anywhere else. [34:19+] Insert "{In subclause 13.5, paragraph 3, insert 'in the initial team' after 'image 1'}." [In a separate J3 paper, I advocate that the kinds of specific discussions in 13.5p3-6 ought to be in the individual procedure descriptions, not in 13.5.] [34:19+] Insert "{In subclause 13.5, paragraph 4, replace 'the interleaving of calls to RANDOM_NUMBER' with 'the sequences of values assigned to the HARVEST argument to RANDOM_NUMBER'}." because the interleaving of calls to all subroutines in unordered segments is processor dependent (although a standard-comforming program cannot observe that), so what's currently in 13.5p4 doesn't say much of anything. [35:24-26] After "present" insert "or is present with the value false", then delete ", otherwise ... specified". [36:6-7] Replace "and" with a comma. At the end of the sentence append ", and the final normative paragraph" [the one added above by the edit at [15:24+]. [36:7+] Insert "{In 13.8.2.16 LOCK_TYPE, in constraints C1303 and C1404, replace 'where' with 'if' twice.}". [36:9] If "5.6" is not generated by \ref, replace "5.6" with "5.8". If it is, check the \ref, and either repair it or run LaTeX a few more times. [36:9+] Subclause 5.8 requires the value of STAT_FAILED_IMAGE to be different from the values of STAT_STOPPED_IMAGE, STAT_LOCKED, STAT_LOCKED_OTHER_IMAGE and STAT_LOCKED, but there are no requirements on the values of the latter constants other than the existing requirement that the value of STAT_STOPPED_IMAGE be different from the value of IOSTAT_INQUIRE_INTERNAL_UNIT. Should that be corrected here, or in the normal course of revising the base standard? If the latter, the TS would need to anticipate the revision and amend it. Therefore, rather than putting the requirements from subclause 5.8 of the TS in a new subclause 13.8.2.21b, append a new subclause 13.8.26 "Uniqueness of values of named constants" with text something like "The values of the named constants IOSTAT_INQUIRE_INTERNAL_UNIT, STAT_FAILED_IMAGE, STAT_LOCKED, STAT_LOCKED_OTHER_IMAGE, STAT_STOPPED_IMAGE, and STAT_UNLOCKED shall be distinct." [36:11+] Insert "{In C1303 in 13.8.2.16, insert 'in an ALLOCATE statement without a SOURCE= specifier, as an in a DEALLOCATE statement' after ''.}" Insert "{In C1304 in 13.8.2.16, insert 'in an ALLOCATE statement without a SOURCE= specifier, as an in a DEALLOCATE statement,' after ''.}" Insert "{In 13.8.2.16 insert a paragraph 'The value of the actual argument associated with the CPTR argument of the C_F_POINTER subroutine from the ISO_C_BINDING intrinsic module shall not be the address of a variable of type LOCK_TYPE, unless the FPTR argument is a pointer of the same type as the object.' }" [36:11+] Insert a subclause 8.9a Edits to clause 15: "{Insert the final normative paragraph of 6.2 at the end of the description of the FPTR argument to C_F_POINTER in subclause 15.2.3.3.}" [36:6-7] Replace "and" with a comma. At the end of the sentence insert "and constraint C604b. Move C604b to be immediately after C435 in subclause 4.5.2.1". [36:11+] Insert "{In 13.8.2.16 LOCK_TYPE, insert 'nonpointer' before 'variable' in C1303 and C1304.}" [36:12+] Insert "{In 16.4 Statement and construct entities, in paragraph 1, after 'DO CONCURRENT' replace 'or' with a comma; after 'ASSOCIATE construct' insert ', or as a coarray specified by a in a CHANGE TEAM construct,'}". [36:14-15] The characteristics of associating entities of the CHANGE TEAM construct are described in 5.3. Replace "and bounds... coselector" with ", bounds, and cobounds specified in [whatever new subclause number of 14-007 subclause 5.3 of the TS becomes]." [36:18+] Insert "{After item (13) in 16.6.7, insert a list item} '(13a) a in a in a CHANGE TEAM construct if the coarray named by the corresponding of that construct [", or a subobject thereof," if 14-136 passes] appears in a variable definition context within that construct;'" [36:23+] Insert "{In the list item concerning COMMAND_ARGUMENT_COUNT etc., insert 'in the initial team' after 'image 1'}". [36:23+] Insert "{In the second list item concerning subclause 13.7, replace 'CMDSTAT or STATUS' with 'CMDSTAT, STAT, or STATUS'.}" Alternatively, change the STAT argument to collective subroutines, EVENT_QUERY, and MOVE_ALLOC, to STATUS, and change nothing at [14-007:468:42]. See also the edits suggested for [36:31+] below, which ought not to be done if the alternative is adopted. [36:25] Include an instruction to insert subclause numbers in the newly-inserted list items. [36:30] Replace with "the relative order of execution of EVENT POST statements in unordered segments;" for consistency with [15:33-34]. [But isn't the relative order of everything in unordered segments processor dependent?] [36:31+] If the STAT argument of collective subroutines, EVENT_QUERY, and MOVE_ALLOC is not changed to STATUS, add three more processor dependencies: " o the value of the STAT argument to collective subroutines if an error occurs (13.???); o the value of the STAT argument to the EVENT_QUERY subroutine if an error occurs (13.???); o the value of the STAT argument to the MOVE_ALLOC subroutine if an error occurs (13.???);" [40:3] Insert "in the initial team" after "image 1". [47:11] Delete "in". _______________________________________________________________________