Martin Packer (Principal z Systems Investigator) and Marna Walle (z/OS Development) are two IBMers talking about whatever z/OS topics come to mind. Often guest experts weigh in on current technologies. This podcast will give you timely, interesting, and entertaining z/OS topics. Each episode comprises a “Mainframe” item, a “Performance” item, and “Topics” which is anything Martin or Marna care to talk about, which might or might not be related to their jobs. So it goes…
Episode 30 "Choices Choices"
35:38Short show notes - long show notes here We have a special guest joining us for the performance topic, Scott Ballentine. What’s New New news for CustomPac removal date, which has been extended past January 2022. The reason was to accommodate the desired Data Set Merge capability in z/OSMF which customers needed. For the new withdrawl date, it is planned to be announced in 2Q2022. Mainframe - Only in V2.5 This topic was looking at some new functions that are only in z/OS V2.5. Data Set File System, planned to be available in 1Q 2022. Dynamic Change Master Catalog RACF data base encryption has a statement of direction. Increase z/OS Memory limit above 4TB to 16TB. More Concurrently ”Open” VSAM Linear Datasets. Db2 exploits with Apar PH09189, and APAR PH33238 is suggested. ISPF Updates to ISPF in support of PDSE V2 member generations, and SUBMIT command to add an optional parameter SUBSYS. Access Method Services - IDCAMS - DELETE MASK has two new options TEST and EXCLUDE z/OS Encryption Readiness Technology zERT There’s a lot of other stuff rolled back to V2.4 Performance - What's the Use? - with special guest Scott Ballentine This discussion is a summary from a joint presentation on Usage Data and IFAUSAGE The topic is motivational because customers can get a lot of value of out this usage data, and understand the provenance of IFAUSAGE data. A macro vendors or anybody use can use to show which products are used and how. It can show names: Product Vendor, Name, ID, Version, Qualifier, Show numbers: Product TCB, Product SRB, FUNCTIONDATA The data is ostensibly for SCRT, which is fed by SMF 70 and SMF 89 Scott talked about encoding via IFAUSAGE, appears in SMF 30 and 89-1 SMF 89-1: Software Levels query, Db2 / MQ subsystems query SMF 30: Topology (e.g. CICS connecting to Db2 or MQ), Some numbers (Connections’ CPU) Both SMF 30 and 89: FUNCTIONDATA Examples were mentioned that might inspire developers to think about how they code IFAUSAGE Hope is developers will delight customers by using IFAUSAGE right, and customers will take advantage in the way shown with reporting examples. Topics - Choices, Choices This topics is about how to choose a language to use for which purpose. Different languages were discussed for different needs. Use Case: Serving HTML? Host consideration: Running on e.g. laptop? Host consideration: Running on z/OS? Overall lesson: Choose the language that respects the problem at hand. On the blog Marna's blog is here Martin has quite a few new blog articles here So It Goes
Episode 29 "Hello Trello"
59:55This episode is about how to be a better z/OS installation specialist, z/OS capture ratios, and a discussion on using Trello. We have a special guest joining us for the performance topic, Miroslava Barahona Rossi. Long show notes are found here . Mainframe - Being a Better Installation Specialist "Better" here means lessons learned, not competing with people. "Installation Specialist" might just as well have used "System Programmer" or another term. There's definitely a reason to be optimistic about being a person that does that Installation Specialist or System Programmer type of work: There will always be a need for someone to know how systems are put together, how they are configured, how they are upgraded, and how they are serviced...how to just get things to run. Consider the new function adoption needs too. There are so many "new" functions that are just waiting to be used. A really good installation specialist would take information they have and use it in different ways. Knowing parmilb really well. You can't be better by only doing things yourself, no one can know everything. You've got to work with others who are specialists in their own area. Last but not least, learn how to teach yourself. Know where to find answers (and that doesn't mean asking people!). Learn how to try out something on a sandbox. Performance - Capture Ratio Our special guest is Miroslava Barahona Rossi, a technical expert who works with large Brasilian customers. Capture Ratio - a ratio of workload CPU to system-level CPU as a percentage. Commercial considerations when billing for uncaptured cycles. You might worry something is wrong if the capture ratio is low. Reasonable value is usually 80 - 90. Seeing more like 85 - 95 these days. It has been improved because more of I/O-related CPU is captured. Summary: Don’t be upset if you get a capture ratio substantially lower than 100%. That's normal. Understand your normal. Be aware of the relationship of your normal to everybody else’s. But, be careful when making that comparison as it is very workload dependent. Understand your data and causes. See if you can find a way of improving it. Keep track of the capture ratio over the months and years. Topics - Hello Trello Trello is based on the Kanban idea. Why are we talking about Trello? We moved to it for high level podcast planning. Template used for hatching new future episodes. We move cards around sometimes between episodes. Marna uses it to organise daily work, with personal workload management and calendaring. But it is not a meetings calendar. Probably with Jira will be more useful than ever. We'll see. Martin uses it for team engagements, with four lists: Potential, Active, Dormant, Completed. Engagement moves between lists as it progresses Martin's blog Marna's blog So It Goes.
Episode 28 "The Preview That We Do Anew"
49:39This episode is about several of the z/OS V2.5 new functions, which were recently announced, for both the Mainframe and Performance topics. Our Topics topic is on Martin's Open Source tool filterCSV. Full long show notes are here. We have a guest for Performance: Nick Matsakis, z/OS Development, IBM Poughkeepsie. Many of the enhancements you'll see in the z/OS V2.5 Preview were provided on earlier z/OS releases via Continuous Delivery PTFs. The APARs are provided in the announce. If you use FTPS for your IBM software electonic delivery, a change is taking place on April 30, 2021. We strongly recommend you use HTTPS instead, but if you still want to use FTPS see IBM software electronic delivery change - take notice! Mainframe - Selected z/OS V2.5 enhancements IBM will have z/OS installable with z/OSMF, in a portable software instance format! z/OS V2.4 will not be installable with z/OSMF, and z/OS V2.4 driving system requirements remain the same. z/OS V2.5 will be installable via z/OSMF, so that is a big driving system change. Learn z/OSMF Software Management at this website. Notification of availability of TCP/IP extended services Workload Manager (WLM) batch initiator management takes into account availability of zIIP capacity Change the Master Catalog without IPL IDCAMS DELETE mask takes TEST and EXCLUDE. IDCAMS REPRO moves I/O buffers above line. New RMF Concept for CF data gathering RMF has been restructured, but all the functions are still intact. z/OS V2.5 RMF is still a priced feature. Performance - zCX enhancements zCX is important for co-locating Linux on Z containers with z/OS. Popular use cases can be found here and in the Redbook here. Another helpful source is Ready for the Cloud with IBM zCX. zIIP eligibility enhancements New enhancements include support 1 MB and 2 GB large pages (still fixed) for backing guests. Guest memory is planned to be configured up to 1 TB Another relief is in Disk space limits Monitor and log zCX resource usage of the root disk, guest memory, swap disk, and data disks in the servers job log. zCX resource shortage z/OS alerts to improve monitoring and automated operations. SIMD (or Vector). SIMD is a performance feature, and can be used for analytics. Nick's presentation (with Mike Fitzpatrick) is here. Topics - filterCSV and tree manipulation Mindmapping leads to trees. Thinking of z/OS: Sysplex -> System -> Db2 -> Connected CICS leads to trees. Has very little automation of its own. But crucially you can mangle the CSV file outside of iThoughts, which is what filterCSV does. filterCSV is a python open source program that manipulates iThoughts CSV files, colouring CICS regions according to naming conventions So it goes.
Episode 27 "And Another Thing"
56:36Episode 27 "And Another Thing" show notes. Long show note are here. Follow up Additional System Recovery Boost enhancements: Sysplex partitioning recovery: Boosts all surviving systems in the sysplex as they recover and takes on additional workload following the planned or unplanned removal of a system from the sysplex. Two more portable software instances: Db2 and IMS added to Shopz on Aug 20, 2020, in addition to CICS (Dec 2019). ASK MPT Martin was asked about which systems in a System Recovery Boost Speed Boost situation get their GCPs sped up to full speed. Mainframe - Trying out Ansible for driving z/OSMF Workflows Marna's been learning about Ansible and how it can drive z/OSMF work. So far with z/OSMF Workflows, from ansible-galaxy collection for ibm.ibmzoszosmf Needed lots of assistance from two gracious Poughkeepsie system testers (massive thanks to Marcos Barbieri and Daniel Gisolfi !!) Learning about an Ansible Encountered two kinds of problems: Security issues connecting the Linux environment to a large Poughkeepsie System Test z/OSMF environment Workflow-related: duplicate instances when part of the playbook ran ok, and selecting an automated workflow. Performance - So You Don't Think You're An Architect? Brand new presentation for 2020, builds on the “who says what SMF is for?” slide in "How To Be A Better Performance Specialist" Presentation talks about 3 layers The Top Layer - Machines, Sysplexes And LPARs The Middle Layer - Workloads And Middleware: Spikes, Restarts, Cloning, Topology, Software Levels The Bottom Layer - Application Components: Db2 and MQ Accounting Trace, DDF topology, data sets Fulfilling the definiton Of Architecture? Understand what's really going on and valuable to system architects Topics - Notifications They are unsolicited messages appearing on the device, generally interruptions whether wanted or not. Examples of what is received and where/ IFTTT still a favorite of Marna's, mentioned in Episode 23 “The Preview That We Do” Topics topic. Martin just subscribed to Premium, but at a low-ball price. How do you manage them to reduce their number. An advanced topic is how you can generate your own On iOS the Pushcut app: a webhook from eg IFTTT can cause a notification On Android we have MacroDroid. Customer requirements Customer Requirement 138729 : z/OS Console message help support JES2 messages IBM response: This RFE has been delivered via APAR PH24072 at end of June 2020. It's available on both V2R3 and V2R4. The JES2 message prefix such as $ can be setup in "Configure Message Help" dialog in z/OSMF Console UI. On the blog Martin's blog posts are here Marna's blog posts are https://blog.share.org/Article/tag/marnas-musings So it goes.
Episode 26 "Sounding Board"
56:39Here are the show notes for Episode 26 "Sounding Board". The show is called this because it relates to our Topics topic, and because we recorded the episode partly in the Pougkeepsie recording studio where Martin sounded zen, and partly at home. What’s New APAR PH21919: NEW FUNCTION - WORKFLOW SUPPORT SAVE JOB OUTPUT PTF Numbers are UI68359 for 2.3 and UI68360 for 2.4 APAR OA56774 (since 2.2) Provides new function to prevent a runaway sysplex application from monopolizing a disproportionate share of CF resources Not based on CF CPU consumption. Is based on deteriorating service times to other structures - which you could measure with SMF 74-4 Coupling Facility Activity data. Mainframe - z15 FIXCATs IBM.Device.Server.z15-8561.RequiredService: Absolute minimum needed to run on a z15. Usually intent is to keep these PTFs to a minimum - and keep the number of PTFs relatively constant. <<Correction: SRB z15 GA1 is here, not Exploitation!>> IBM.Device.Server.z15-8561.Exploitation: Needed for optional functions, and you can decide when you want to use them. This PTF list could grow - if we add new functions IBM.Device.Server.z15-8561.RecommendedService: Usually to fix a defect that is found but haven’t risen up to required. Might want to run an SMP/E REPORT MISSINGFIX to see what’s in this FIXCAT. Might install some, all, or none of the fixes. Might want to be more selective. Based on how much change you want to encounter, versus what problems are fixed By the way there are other FIXCATs you might want to be interested in for z15, e.g. IBM.Function.SYSPLEXDataSharing Performance - DFSORT And Large Memory A very special guest joins us, Dave Betten, former DFSORT performance lead. Dave talked about DFSORT memory controls DFSORT has historically been an aggressive user of memory Installation defaults can be used to control that But the EXPOLD parameter needs special care - because of what constitutes "old pages", which aren't actually unused. DFSORT Tuning Guide, especially Chapter 3 Dave talked about how handy rustling up RMF Overview Reports can be, with several Overview conditions related to memory. Most of the information in this topic is relevant to LPARs of all sizes Topics - Update on recording techniques Recording of podcast - still using Skype to collaborate Production - Editing - Moved to Ferrite on iPad OS Publishing - same process as before, but on a new publisher! Customer requirements (assume usual disclaimer) RFE 139477 “Please include the CPU Time Limit for a Job/Step in SMF Type 30” After the fact - End of step looks useful. Could run query comparing to actual CPU time, then track to see if ABEND is on the horizon “ As it happens ” - Would like on the SMF Interval as well as Step End records. May not need the SMF information if vendor and IBM tools already do it today. On the blog Martin's has blog posts since the last episode. Marna has three on her blog here. So it goes...
Episode 25 "Flit for Purpose"
41:02Here are the show notes for Episode 25 “Flit for Purpose”. The show is called this because it relates to our Topic, and also can be related to our Mainframe topic (as a pun for “Fit for Purpose”). Mainframe Topic: Highest highlights of z/OS V2.4 and z/OS on z15 Highlight 1: zCX Highlight 2: z/OSMF Lots of z/OSMF enhancements that have arrived in z/OS V2.4, and the good news is that most of them are rolled back to V2.3 in PTFs that have been arriving quarterly. Security Configuration Assistant: A way within z/OSMF to validate your security configuration with graphic views, on the user and user group level. Designed to work with all three External Security Managers! Available back to V2.3 with APAR PH15504 and additional group id enhancements in APAR PH17871 Diagnostic Assistant for z/OSMF : A much simplier way to gather the necessary information to need for a Service person to perform debug for your z/OSMF problem. Hightlight 3: SRB on on z15: System Recovery Boost :Speeds up your shutdown for up to 30 minutes and speeds your re-IPL for 60 minutes, with no increase to your rolling four hour average. Performance Topic: z15 from chip design on upwards Disclaimer: personal view, not from Development or Marketing. Marna and Martin were talking about the z15 Chip design – and we thought those observations might be useful to include in the Performance topic. Two traditional levers were raising clock speed or shrinking the feature size. GHz and nm aren’t the be all and end all. Look at chip design. Start with a similar sized CP chip and putting more on it. It helped to get rid of the Infiniband-related circuits, and some layout enhancements. At the top end there are up to 190 characterisable cores, coming up from 170. This can give us a fifth drawer – which is quite important. Topic: How To Do A Moonlight Flit This topic is about moving one’s social output, in particular blogs and podcast series. Martin’s blog had to move, because the IBM developerWorks blog site is being shut down. Martin's blog has had to move, as well as this podcast. Immediately people might worry about Request For Enhancements being affected , and it is not. Martin and Marna discuss important aspects to consider when moving your social media. You must consider all the pieces when do you the most. You must also try to redirect your audience. Contacting us You can reach Marna on Twitter as mwalle and by email. You can reach Martin on Twitter as martinpacker and by email and blogs at blog.
Episode 24 "Our Wurst Episode"
42:34Here are the show notes for Episode 24 “Our Wurst Episode”. The show is called this because we both attended the IBM TechU in Berlin, Germany, and our Topics topic is our trip report. Mainframe Topic: CICS ServerPac in z/OSMF IBM’s first delivery on new installation strategy, will be with CICS and associated SREL products. This is the first of many (really, all). Choice on new installation strategy or old during ShopZ ordering. We encourage making the z/OSMF choice, as that is consistent between IBM and other vendors, and is intended to be easier. Infrastructure already available in continuously delivery PTFs, and rolled back to z/OS V2.2. This makes the driving system have the proper infrastructure so anybody can package and deliver that way. All software that you ordered as a ServerPac, and installed either way, will give you the same (or hopefully better) equivalent installation. There is an IBM Statement Of Direction that this installation choice is coming, but we do not have an exact date yet. For other software ISVs, they can exploit the new z/OS installation strategy whenever they are ready. Prepare now by becoming familiar with z/OSMF Software Management and Workflows Performance Topic: DB2 And I/O Priority Queuing Follow on from Screencast / Martin's Blog post topic:, "Screencast 12 – Get WLM Set Up Right" Recent talk has been about whether to turn off I/O Priority Queuing in WLM. Service classes with DB2 subsystems in are heavily I/O Sample oriented, which is unusual among service classes in a system. Means access to CPU is not properly managed, as CPU & zIIP samples few, relative to I/O samples. Reminder: Most of DBM1 is now zIIP-eligible. Can achieve goal even with lots of delay for zIIP or CPU, but that’s definitely not what you want. You don’t want to just turn off WLM I/O Priority Queuing, as it’s sysplex-wide, it might affect other work that needs it, and Db2 might actually need it. First you need a reasonably achievable goal for the service class. Make sure you’re more or less achieving the existing goal. Second, calculate what the velocity achieved would be without I/O priority queuing . You should evaluate or adjust the goal attainment, but that is BAU. Changing WLM always needs some care. Topics: Berlin Trip Report May 20-24 We both attended IBM Z TechU in Berlin, and got to see each other. Marna had about six sessions. The SMP/E Rookies session had fabulous attendance – 44. Some were more experienced, but most were not. z/OSMF had good attendance too, about 82. More are interested in this topic, especially if you compare to just a couple of years ago. Best attended was the z/OS V2.4 Preview, with about 150 people. There was excellent interest in what is coming in the new release. Marna did her own poster to help with z/OSMF configuration, and several people came by to chat. Both Marna and Martin shared a poster about this podcast. We helped with getting one person a podcast app (on each platform), and a subscription to this podcast. Martin had five sessions. One was co-written with Anna, “2 / 4 LPARs” Two were solo efforts: Parallel Sysplex Performance Topics, and Even More Fun With DDF. Contacting Us You can reach Marna on Twitter as mwalle and by email. You can reach Martin on Twitter as martinpacker and by email and blogs at blog.
Episode 23 "The Preview that We Do"
33:10Here are the show notes for Episode 23 “The Preview That We Do”. The show is called this because we talk about the newly previewed z/OS release, V2.4, in the Mainframe section. This is our 24th episode too! How convenient! Mainframe: z/OS V2.4 Preview “z/OS Container Extensions” (zCX) zCX Information z/OS Upgrade Workflow, no book More in Pervasive Encryption zFS enhancements MCS logon passphrases Biggest question one may have: what level of HW will z/OS V2.4 IPL on? z/OS V2.4 will run on zEC12/BC12 and higher. Performance: Coupling Facility Structure Duplexing Two types of CF structure duplexing: User-Managed: Only DB2 Group Buffer Pools (GBP) System-Managed: e.g DB2 IRLM LOCK1 Structure Structure performance matters: User-Managed not an issue. System-Managed matters. Asynchonous CF Structure Duplexing Announced October 2016 Important considerations if Async CF Duplexing good all the time: People make architectural decisions and this should not be a leap in the dark . Ideally should be established with a little testing, with testing as close to production behaviors as possible. Generally it’s good for you. Configuration: Format couple data set, put into service, and then REALLOCATE. Again speaks to planning and testing. The main event for this item is SMF. SMF 74-4 Coupling Facility Activity data, primarily interested in structure-level, especially for structure duplexing of any kind. Though CF to CF pathing information also available. New Async CF Duplexing instrumentation APAR OA49148 Early days of Async CF Duplexing despite having been announced in 2016. Martin has been using a customer’s test data, and would like to build experience. Only a portion of this new SMF 74-4 data is surfaced in RMF Postprocessor reports. z/OSMF Sysplex Management can help visualize and control the Sysplex resources. This function to help with control is in PI99307: SYSPLEX MANAGEMENT APPLICATION ENHANCEMENTS TO MODIFY SYSPLEX RESOURCES. Topics: Smart home thermostats Marna just installed two Nest thermostats, one in each zone (of a three-zone house). Is sharing data with Nest, and presumably whoever owns Nest currently (Google). Instrumentation benefit is having an app in which she can look at what is happening at home, when away, and control it too. There are excellent graphs on what has been used (hours of heating, cooling) in the app. Martin doesn’t have a smart home thermostat, but does have a remote oil tank sensor to determine how much oil is left. This sensor feeds back into a device in the house, and connects to an app on his phone. It costs 5 GBP a month, but is unsure yet if it is worth it. Contacting Us You can reach Marna on Twitter as mwalle and by email. You can reach Martin on Twitter as martinpacker and by email and blogs at blog.
Episode 22 "Great App-spectations"
42:08Here are the show notes for Episode 22 “Great App-spectations”. The show is called this because we talk about app expections in our Topics topic. Mainframe: PI99365 Two enhancements in z/OSMF Operator Consoles Support for “sticking” WTOR and held messages on the top of the console area Visible EMCS console name View WTOR and HOLD messages in a separate window Visible console name part Overall: These two function areas help you manage your z/OSMF operator consoles better. Performance: Paging Subsystem Design in an age of Virtual Flash Performance: Paging Subsystem Design in an age of Virtual Flash Question from customer about need for paging space if Flash installed , which was answered in Martin’s blog post, but there is more thinking about this. Look at the paging subsystem design in the round, with two flavours of Flash: FlashExpress (PCIE) and VFM Design standpoint ideally as if no Flash Paging subsystem design: Two main considerations: Space and Performance Come together in “30% Contiguous Slot Allocation Algorithm breakdown” rule of thumb Instrumentation RMF SMF 71, also covers memory. Flash treated like ordinary paging device. RMF SMF 74-10 in OA50761 Wrap up: Paging subsystem design still worthy of care, and establish whether risk of Flash or Virtual Flash warrants conservative configuration of paging subsystem. Topics: Anatomy Of A Great App “App” here means “third party software” but we’ll say app for short Highly biased on expectations in iOS, as Martin is a power user. Marna gave the Android perspective z/OS perspective was discussed. Conclusion: Think about more than just what your app is supposed to do. Nobody wants software whose function they like but they hate using. It is way too easy to uninstall an app (or have hundreds of them and not use them). Keep to the “Principle of least astonishment”. Contacting Us You can reach Marna on Twitter as mwalle and by email. You can reach Martin on Twitter as martinpacker and by email and blogs at blog.
Episode 21 "Fits and Starts"
33:17Here are the show notes for Episode 21 “Fits and Starts”. The show is called this because we talk about fitness devices, and the Performance topic that had work submitted after a one minute hiatus. Mainframe: SMF Recording of APF Modifications Post-IPL dynamic APF changes are reflected in SMF 90 Subtype 37. A lot of the function is in z/OS V2.2, with these fields in the SMF record: Function: Add, Delete, DynFormat, StatFormat Was the update via SETPROG, SET PROG, CSVAPF Parmlib member suffix for the SET PROG case, ... Triggers when post-IPL APF changes dynamically: PROGxx: APF ADD …or APF DELETE … SETPROG APF,ADD,… or SETPROG APF,DELETE,… Ensure to collect by setting in SMFPRMxx type 90 subtype 37 record Presumably there’s not much overhead, as it will be produced when changes happen (which is probablyl not often). Auditors will probably want this Performance: An interesting Db2 DDF case Central to Martin’s DDF work is some analysis code to process SMF 101 DB2 Accounting Trace. A customer complained their DDF application stopped dead one evening – for 1 minute. It was an application serviced by a 3-way Datasharing group. The customer sent SMF 101 data from all 3 members for 3 hours around the stoppage, and for 3 hours the previous evening for a presumably “good behaviour”. Martin plotted application statistics at a one second interval level. It showed a 40-second stoppage the evening they hadn’t complained, making the 1 minute threshold interesting as a number. Martin “zoomed in” to a much shorter time range . When transactions started again they were elongated, and it that was due to the clustered arrivals in clearing the backlog. The best theory is something external stopped transactions arriving. Further he thought there could be “near misses” many times, just short of the 1 minute mark. After transactions started coming again there were spikes in transactions arriving every minute. The speculation is this might be the middle tier doing something on a 1 minute basis: Maybe retries of some sort? Topics: Fitness Tracking Marna uses a Fitbit Charge 2. Key features: sleep analysis, step counts, heart rate. Fitbit app for the Android: calculates floors, miles, calories, sleep analysis, across timescales – day, month, overall, etc and compares with age bracket. Martin uses an Apple Watch, and used to have a Fitbit. He wanted the Watch for other reasons: for health a few months ago, and all of the above – except for sleep tracking. Marna gets employer incentives, to help with health care cost reduction. Martin has been successful, as he hasn’t failed to close his rings for 3 months, for the Apple Watch. This makes him obsessive. Uses iOS Overcast podcast player to send podcast episodes to the watch. Runs with just the watch and AirPods. Listening to podcasts keeps me going – whether running or walking. He has lost a considerable amount of weight! Cardiac situation much better, lowered resting heart rate, faster recovery from exercise, and clothes fit better. Contacting Us You can reach Marna on Twitter as mwalle and by email. You can reach Martin on Twitter as martinpacker and by email and blogs at blog.