Simon Griffee Either do something very useful, or very beautiful.
July 2013
Goodbye, J.J. Cale
You’ve been with me at home and on the highway. Thank you (Mike Johnston on J.J. Cale.).
dream - sunday, 07 july 2013, 05:38:37
driving by fenced field with vegetable gardens and reeds, i know the field was wild before, and someone has been shooting animals, bodies of a fox and another animal hanging on a barbed wire fence. a horse and a goat(?) are watching me pass, their heads over the fence next to the dead animals, can’t see whether they are attached to the fence or not, by the skin on the top of their heads. car is beeping, fuel gauge shows almost empty, there is a sort of metallic plastic foil over part of the steering wheel. now inside, walking into empty house, light swtiches don’t work, realize i haven’t spoken to or seen dad for a long time. begin calling out louder, dad, dad, dad, dad, dad! and i hear his cough, and he is there, in front of the computer in a corridor. he is putting super-glue on his eye to try and keep it closed. no! you can’t do that dad! but he doesn’t care, he wants the eye, the right eye, to close, and he puts the glue on it, on the eyelashes and the eyeball. his face now looks better and i tell him he looks good. now he suddenly does look good, younger, a young english actor look, jeans, wet, combed hair to the side. ‘i’m in contact with someone for work’. he says. i reply, ‘i can help you look for work, dad.’ he says ‘i may not be looking for work with whom you think’.
EuroPython 2013 Notes
Rough notes from EuroPython 2013.
The Next 20 Years of Python — V. Van Lindberg
Chairman of the Python Software Foundation, works for Rackspace and OpenStack.
- Opinionated language.
- Guido managed to build a community. Sense of values.
import this. - Zen of Python == core part of ethos.
- Standards language for bioinformatics.
- Primary teaching language for most universities, finally replacing Java (thank goodness).
- Community needs to scale.
- Raspberry Pi, Upton says ‘Pi’ stands for Python.
- The prompt as first thing you see when booting a computer is a powerful concept. Today computers primarily used for consumption. Consumption-oriented culture.
- Young coders, education, is very important. How to use computers to create. Young coders curriculum will be available in new http://www.python.org (python.org built with Django).
Thinking Outside the Box — Armin Ronacher
Creator of the Flask framework and the Werkzeug and Jinja2 libraries among many others
We all suffer from going with the flow in many cases.
The keynote gave some examples of how questioning some established rules revolutionized the industry and how we open our mind a bit.
It included interesting examples from the Python and programming community where people tried something new with tremendous success and how they came up with the idea.
- Community influences you.
- Ask right questions, eg. “I don’t want my users to wait while I process data.”
- Assume you’re already wrong.
- Describe the problem, leave room for answers.
- http://bit.ly/pypush
- Go on IRC.
- Most things have a design behind them.
- Gets lost as copied (Chinese whispers).
- Original design may no longer apply.
- Look up wheel package management.
- Paradigm shift as critical mass of people choose a way to do things.
- Sometimes all that is necessary is to transport idea from one industry to another.
The Return of Peer2Peer Computing — Holger Krekel
Founder and co-developer of a number of popular Python projects, among them PyPy, py.test, tox and execnet. Regularly gives talks at conferences as well as open courses and in-house trainings on testing topics. His interests also include P2P computing, metaprogramming and politics.
A video of this presentation is now online.
Consider myself from global IT community, not from a particular nation
@gonzohacker:
Someday when I’m living in a post-apocalyptic police state, run by autonomous flying drones, I want to tell my kids, “I helped build that.”
How much is Google, Facebook, etc. responsible for aiding Spying Cartel?
Which Internet Future do we prefer:
- Greatest freedom catalyst ever?
- Greatest surveillance machine ever?
Which one happens depends on:
- Politics & laws.
- Technologies.
IT people are like bishops. We know how programming works and relates to society. We have a responsibility to make choices that cause no harm. It’s all about communication between devices, mediated by technology.
Centralization (current situation)
0 0
\ /
0 — 0 — 0
/ \
0 0
Espionage Cartels are happy:
- Communication mediated by few companies.
- Companies can be compelled to co-operate with government.
- Weak or no end-to-end encryption == BIG PROBLEM.
How to disrupt Cartels?
Decentralization
0 — 0 — 0 0 — 0
/ \ / | / \ /
0 — 0 — 0 — 0 — 0
/ | /
0 0 — 0
- End-to-end encryption.
- P2P routing topologies.
- Distributed mediation ownership.
Example: http://crypho.com, but: crypho as mediator has meta-data == PROBLEM
P2P code needs to be FOSS & run on OWN device.
Example: TAHOE-LAFS storage:
- Provider-independent security.
- Decentralized, redundant, distributed.
- Active FOSS (free, open source software) Python project.
Communication is about more than only file storage. New promising WC3 standard: Web RTC - browser-to-browser data streaming, real-time communications.
With power comes abuse
We are good at client/server, not so much P2P"
Global WLAN situation is idiotic. Idiotic == private ‘only for me’ in ancient greek
Lookup:
- http://ozan.io/p/ : Peer-to-peer networking with browsers.
- http://sharefest.me : Browser-based P2P filesharing using WebRTC.
- http://vole.cc : P2P social networking project.
- http://nightweb.net : P2P social networking project with bittorrent.
- WebCrypto API - native encryption in browser.
- Project SPAN: ad-hoc android mesh networks.
- http://Freifunk.net = free WLAN network.
- Bittorrent Sync - Automatically sync files via secure, distributed technology. Great free, open source alternative to Dropbox.
- Bittorrent: written in Python, 10% of total internet traffic.
Given enough time, looking at history, countries are in danger of having criminals in charge.
It’s about Convenience. Some choose inconvenience (Edward Snowden).
¿Q: Is there hope for a world without nation states?
Yes. FOSS is transnational. WebRTC standard - nations had no say. Nations not so important in internet world.
How do we do mediation?
-
Need social ID
-
TAHOE-LAFS trust model works.
It would make much more sense to have government communications open and transparent as they have nothing to hide, right?
Answer to the “If you have nothing to hide, you shouldn’t worry about surveillance” argument: The damage from a totalitarian society is much higher than from terrorists.
“Good Enough” is Good Enough! — Alex Martelli
Alex Martelli in Florence in 2013.
Author of Python in a Nutshell and the first 2 editions of the Python Cookbook. For the last 8+ years has been working (as ‘uber’ tech lead and senior staff engineer) for Google in Mountain View, California.
“Worse is Better” —Unix, ‘New Jersey’ approach.
- Simplicity.
- Implementation & interface.
- Most important consideration is design.
- Correctness.
- Slightly better simple than correct.
- Consistency.
- Not overly inconsistent.
- Completeness.
- Can be sacrificed to any of above 3.
- MUST be, if simplicity is threatened.
Well == Simple.
G.K. Chesterton:
Anything worth doing is worth doing badly.
E. Raymond:
Given enough eyeballs, all bugs are shallow.
Perfection == Big Design Up Front.
Alex Martelli:
The world is changing all the time. Your perfection is a memory of the past.
Use ‘Perfect’ as Verb, rather than adjective.
Don’t save on:
- Light-weight agile process and its steps.
- Revision control, code reviews, testing.
- Proper release-engineering practices.
- Code style, clarity, elegance.
- Documentation.
No cowboy coding!
From the start:
- Security.
- Privacy.
- Auditability.
Best at start, but acceptable to add later:
- Modularity, plugins.
- API (application programming interface).
- Scalability.
General vs ad-hoc solution
def find_by_col(root, color):
if root.color == color: yield root
yield from (find_by_col(c. color) for c in root.cs)
def find_by_sha(root, shape):
if root.shape == shape: yield root
yield from (find_by_sha(c. shape) for c in root.cs)
Versus:
def find(root, n, v):
if getattr(root, n) == v: yield root
yield from (find(c, n, v) for c in root.cs)
Okay to incur technical debt, WITH CARE.
- Eg: n log n vs. n2
Focus on potential errors that could cause irrecoverable losses:
- As long as you can recover, it’s okay.
- In ‘beta’ at least.
- Is reputational damage to yourself recoverable?
- Depends, but most usually, yes.
- Especially courteous, speedy response to issues that get reported = service.
- Getting it right the 2nd time is usually OK.
- Hiring: you have to pick the best one who will have you.
- No one cares about perfect lines - Am I moving something in people’s hearts?
- Keep it simple"
- Gettysburg Dedication
- the “Oration”: the soon-forgotten-one
- Edward Everett
- 13,508 words, two hours, reams of paper
- and then…
- Gettysburg Address
- Abraham Lincoln
- 267 words; two minutes, back of envelope
- “the world will little note, nor long. remember what we say here…”
- But, 150 years later, it usre still DOES!-)
- Gettysburg Dedication
Solid Python Application Deployments for Everybody - Hynek Schlawack
Hynek Schlawack is an infrastructure and software engineer from Berlin, Germany, PSF fellow and contributor to a wide variety of open source projects.
How a German web hosting company and domain registrar deploys numerous Python applications on UNIX-based systems. Simple is not Easy. Put effort into making your deployments simple.
Edsger W. Dijkstra:
Simplicity is prerequisite for reliability.
Donald Knuth:
It is important to find simple solutions instead of stopping as soon as a first solution is found.
- Package
- Use Vagrant.
- Application is tied to server OS version.
- Upgrading servers == updating your app.
- Packaging such as .rpm, .deb even .pkg.tgz, then:
- checkout from VCS.
- create virtualenv.
- install dependencies.
- do whatever you want.
- package the result.
- push to your repo.
- Packaging is not hard. Use fpm.
- Automate - use configuration management, test in staging.
- Security - Be paranoid, iptables, /bin/false, ssl, fail2ban, file sockets, revoke all, etc.
- Restart - upstart, systemd, supervisord, circus, etc.
- Log - log to stderr, redirect stderr syslog, use OS tools.
- Deploy, using Apache is fine if conscious decision is made to use it, but uWSGI or Gunicorn + Nginx provides better separation of concerns and is simple to set up and has some awesome benefits (uWSGI emperor mode).
- Monitor - nagios, pingdom, etc.
- Measure - graphite, statsd, etc.
Links:
- http://hynek.me/talks/python-deployments/
- https://speakerdeck.com/hynek/solid-python-application-deployments-for-everybody-ep-edition
Introduce Django to Your Old Friends - Lynn Root
Lynn Root is a software engineer at Red Hat on the Identity Management team. She is the founder and leader of PyLadies San Francisco, an international mentorship group for women and friends in the Python community.
How to integrate Django applications to authenticate against already existing identity management systems like LDAP, Kerberos, etc. Links:
- http://www.roguelynn.com/kerberos/
- https://speakerdeck.com/roguelynn/europython-2013-introduce-django-to-your-old-friends
- http://chirale.wordpress.com/2013/03/15/unified-login-in-django-and-drupal/
Sink or swim, five jackets to throw to the new coder - Lynn Root - http://newcoder.io
Advanced e-learning, or introducing programming to new coders.
- Software development moves fast. How to teach programming?
- Concepts.
- Episodic vs. cumulative learning.
- Project-based learning.
- Learn to tread water by playing water polo, not by someone dictating how to move arms & legs.
- Knowledge is actively constructed by the student.
- Build things that are used in the real world.
Tutorials:
- Dataviz - <http://newcoder.io/dataviz/
- APIs - http://newcoder.io/api/
- Web scraping - http://newcoder.io/scrape/
- Networks - http://newcoder.io/networks/
- GUI (soon)
Intro to Django - Daniel Greenfeld & Audrey Roy
Daniel Greenfeld is a software developer and co-author of Two Scoops of Django. Audrey Roy is a software developer and designer and co-author of Two Scoops of Django.
Hands-on Django workshop. Built a real, working site from the ground up, using Django 1.6 and Python 2.7/3.3. Covered Python and Django setup, project creation, app creation, models, the Django admin UI, views, migrations with South, user image uploads, new user registration, basic forms, and basic internationalization.
Introducing Python as a Main Programming Language in a Company - Patrick Büchler
Patrick Büchler is a Software Developer.
Car analogy - people who go to BMW will get different quality than Fiat. Same with software, but not appreciated.
Right mix of people important. Pointless to have lots of all-stars because they will all have their own idea of how it should be done.
Why Python? Too many languages == uncomfortably hot. Python is platform independent, has big community, good traction/adoption/future plans, good learning curve:
- Simple and readable
- Versatile and platform agnostic
- Can be used for web and on client
- Also a philosophy, i.e. the Zen of Python:
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.
Now is better than never.
Although never is often better than right now.
If the implementation is hard to explain, it’s a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea – let’s do more of those!
How to implement Python? How to manage risks?
- If you take a decision, implement it quickly.
- People learn only with real challenges.
- Innovation not the same as research. As entrepreneur you must innovate, not research.
- Can’t afford to spend much money on not-financed activities.
Choose project and do it with Python
- Project should not be critical.
- Not be too small (at least 1 man-year).
- Should not contain complex technical challenges.
- Should not contain complex business logic.
- Should not be in a risky domain.
- Should use easy to use framework (such as Django or web2py)
- Should be assigned to a specific team, without people who are against the project, with mix of first-movers and conservatives.
- Primary goal should still be to finish the project with a profit.
Problems
Mostly went surprisingly smoothly! Consider the following:
- Python is not Java!!
def my_function() != public int myFunction()Do code reviews! Especially when devs used Java or C# before. Developers coming from VB have less projects. - Allow refactoring: give your team the time to make their first project good.
- Enforce refactoring: force your team to make the first project good.
- Be patient: do not forget that it is new for most developers.
- Be strict: you decided to risk implementing the project in a new language, stick to your decision.
- Python is agile, use this: with python debug cycle is very quick, use this i.e. in sprint review.
- Don’t interfere too much: let your colleagues work on their own, so they learn!
Factors critical for decision about tech to use:
Tech and org factors (reasonable)
- Surrounding systems enforce specific tech.
- Base systems used enforce particular tech i.e. Oracle.
- IT Departments often proficient on one platform only.
- Contracts with vendors or suppliers enforce given tech.
- High investments required to change.
Human Factors (personal)
- IT Departments often reluctant to change their systems even when outdated or try something new (consider the type of personality who becomes sys admin).
- Management hear saying (we MUST have SharePoint because it will solve all our problems as my friend told me on golf course).
- Internal Politics (we decided that XYZ is the best architecture, we cannot change now).
- Quasi-religious beliefs or important decision makers or influential people (there is no better sport than to provoke a dev by criticizing his language of choice).
The word ‘Python’ won’t open doors, but the folowing may:
- Platform independent.
- Very fast development cycle - you can even debug and change the code onsite during the sprint review!!
- Completely open source - not owned by Oracle unlike Java, MySQL, etc.
- Mature.
- Big community.
Fears/questions from customers:
- Fear of Vendor Lockin: most business people know .NET and Java and believe all else a niche.
- Fear of Open Source: understandable, many dead open source projects.
- Lend helping hand to sysadmins: maybe he used SQL Server and IIS since 199x! Suddenly it’s PostgreSQL and web2py. Help him and he will be a grateful promoter of your technology.
- Most problems not technical! Remind your customer and your team about this.
Was it worth it?
:) Yes!
- Python is well established now
- At moment 4 different projects with Python
- Strategic decision: all individual web projects are done with web2py
- Wherever possible suggest using Python
Q: Tried sales pitch “lets do a prototype in Python first and see if it works” A: No, but it is a good idea
General notes:
- Education needs to have a practical purpose, i.e. learn by working on a project
- Good to point out all the great software already available in Python, i.e. 1 line instead of 25 lines - http requests
- If you run into performance issues, do code review, use C and bind to python, etc.