PACE Advisory Committee Meeting - May 27, 2021

PAC Meeting Summary, May 27, 2021

Proposal regarding credit expiration reviewed by PACE team, and the following has been approved:

Existing compute credits from the Rich Building migration, which were originally set to expire in 5 years (April 2026), will remain available for 7.5 years (October 2028). If there are still credits available at that time, PIs may request a one-time no-cost 1 year extension.

After checking with Grants and Contracts Accounting and EVPR office, the following has been approved: 

PACE credits can be used for cost share arrangements. See for further details on how this should be handled.

Discussion of other expiration concerns, in particular startup and retention. Current practice is that College and EVPR split PACE startup and retention commitments. CoS proposes to have no expiration on their half of the contribution. EVPR commitment would remain at 3 years and ideally spent first. Follow up with other Deans to see if they want to consider something similar.  PAC may suggest an updated model for EVPR half after the current Provost review of startup practices is completed.

For future discussion: Propose a consistent model for the Institute where all startup and retention funds would be available for 7.5 years with 1 year no cost extension (like the current Rich building equipment credits). This would create a consistent “floor” which allows startup and retention funds to more closely compare with an equipment purchase. CoS may still choose to keep their half unexpired.

Discussion of PAC communication to the larger community. Use PACE website area. 

Consider changes to PACE advisory committee makeup.  Decided to leave as is, but make sure PAC activity is communicated more broadly.  Also, if unit IT Directors are finding that the new model is increasing the number of ad hoc clusters, then PAC should be engaged to understand the situation and address any gaps in PACE services.

The F&A waiver on PACE charges will include consulting charges (this wasn't clear).  Will be reviewed in Spring 2023, when we have more data and at the same time the full F&A waiver is reviewed. PIs who request sponsored funds for consulting charges should handle them in the same section in their budgets (M&S) as other PACE compute & storage charges.  In particular, they should NOT be included as personnel lines.

Discussion of CoS memo.  General support for the relevant well-articulated concerns.  Most are already captured in our working list.

Some recent CoS faculty startup was promised to purchase equipment (presumably in Rich) but ended up being converted fully into credits. These faculty (list to be provided) will be given the better of two options:  7.5 years of credits (with 1 year no cost extension option) or, since they are CoS, 50% of their credits never expire.  The latter option is default. 

Announcements and discussion topics:

  • PIs can now optionally put guardrails, soft and hard limits on usage. 
  • Dashboard for tracking usage.  Still in development but available soon. It would be nice to include the cost of nodes in the dashboard.
  • Increase Flexibility (reducing one-size-fits-all approach): sharing of credits among small to medium sized groups now widely available through hierarchical credits model.  Needs to be put on website.
  • Increase Flexibility (reducing one-size-fits-all approach): Use reservation system to guarantee certain capacity for benchmark runs, etc.  Note reservation implies you pay 100% of cost, whether used or not.
  • If wait times exceed certain thresholds, (e.g., In x weeks y % of jobs have more than z hours wait) then time to get more nodes of appropriate type.  Option to also fence off dedicated resources temporarily.
  • Scheduling downtimes before a big conference is a problem. Maybe use reservations to see when to schedule downtime.

Other topics:

Seek balance between HPC and HTC and optimize it.

Traffic on Phoenix (I/O can be swamped by HTC).

Discussed future meeting schedule. June is already scheduled. Skip July, and continue monthly through December.  Revisit in November.